Original: Roadblocks for Temporarily Disabling Shortcuts and Learning New Knowledge
Revised: Roadblocks for Temporarily Disabling Shortcuts and Learning New Knowledge

Original: Anonymous Author(s)

Original: AfﬁliationAddress email

Original: Abstract
Revised: Abstract

Original: Deep learning models have been found with a tendency of relying on shortcuts, i.e.,decision rules that perform well on standard benchmarks but fail when transferredto more challenging testing conditions.
Revised: Deep learning models have been found with a tendency of relying on shortcuts, i.e., decision rules that perform well on standard benchmarks but fail when transferred to more challenging testing conditions.

Original: Such reliance may hinder deep learningmodels from learning other task-related features and seriously affect their performance and robustness.
Revised: Such reliance may hinder deep learning models from learning other task-related features and seriously affect their performance and robustness.

Original: Although recent studies have shown some characteristics ofshortcuts, there are few investigations on how to help the deep learning models tosolve shortcut problems.
Revised: Although recent studies have shown some characteristics of shortcuts, there are few investigations on how to help the deep learning models to solve shortcut problems.

Original: This paper proposes a framework to address this issue bysetting up roadblocks on shortcuts.
Revised: This paper proposes a framework to address this issue by setting up roadblocks on shortcuts.

Original: Speciﬁcally, it can automatically make modiﬁcations based on the original task.
Revised: Speciﬁcally, it can automatically make modiﬁcations based on the original task.

Original: Roadblocks are placed during the modiﬁcationprocess to ensure that the learned knowledge, including shortcuts, is insufﬁcientto complete the modiﬁed task.
Revised: Roadblocks are placed during the modiﬁcation process to ensure that the learned knowledge, including shortcuts, is insufﬁcient to complete the modiﬁed task.

Original: Therefore, the model trained on the modiﬁed taskwill no longer over-rely on shortcuts.
Revised: Therefore, the model trained on the modiﬁed task will no longer over-rely on shortcuts.

Original: Extensive experiments demonstrate thatthe proposed framework signiﬁcantly improves the training of networks on bothsynthetic and real-world datasets in terms of classiﬁcation accuracy and featurediversity.
Revised: Extensive experiments demonstrate that the proposed framework signiﬁcantly improves the training of networks on both synthetic and real-world datasets in terms of classiﬁcation accuracy and feature diversity.

Original: Moreover, visualization results show that the mechanism by which ourmethod works is consistent with our expectations.
Revised: Moreover, visualization results show that the mechanism behind the proposed our method is consistent with our expectations.

Original: In summary, our approach caneffectively disable the shortcuts and thus learn more robust features.
Revised: In summary, our approach can effectively disable the shortcuts and thus learn more robust features.

Original:  Introduction
Revised:  Introduction

Original: Deep learning has triggered the current rise of artiﬁcial intelligence and is the workhorse of today’smachine intelligence [13].
Revised: Deep learning has triggered the current rise of artiﬁcial intelligence and is the workhorse of today’s machine intelligence [13].

Original: One of the great advantages of deep learning is the ability to automaticallyextract the required features according to the task.
Revised: One of the great advantages of deep learning is the ability to automatically extract the required features according to the task.

Original: And the massive amount of data that drives themodel is the key to the success of deep learning.
Revised: And the massive amount of data that drives the model is the key to the success of deep learning.

Original: Regrettably, deep learning methods can hardly takefull advantage of the training data, and an important reason is over-reliance on simple features.
Revised: Regrettably, deep learning methods can hardly take full advantage of the training data, and an important reason is over-reliance on simple features.

Original: Deeplearning methods have recently been found to have a tendency to learn simple features only [4].
Revised: Deep learning methods have recently been found to have a tendency to learn simple features only [4].

Original: Thelearned simple features may be sufﬁcient for the accessible data (training data) and the model willhave difﬁculty learning other complex but effective features.
Revised: The learned simple features may be sufﬁcient for the accessible data (training data), and the model will have difﬁculty learning other complex but effective features.

Original: Insufﬁcient learning of effective features may hurt accuracy and generalization performance.
Revised: Insufﬁcient learning of effective features may hurt accuracy and generalization performance.

Original: Worseyet, when the sample distribution changes and the learned simple features are no longer valid, deeplearning models may collapse since they did not learn other features.
Revised: Worse yet, when the sample distribution changes and the learned simple features are no longer valid, deep learning models may collapse since they did not learn other features.

Original: Therefore, suppressing thereliance on simple features is indispensable for designing models.
Revised: Therefore, suppressing the reliance on simple features is indispensable for designing models.

Original: Shortcuts in deep learning are described as the decision rules that perform well on standard benchmarks but fail to be transferred to more challenging testing scenarios according to [4].
Revised: Shortcuts in deep learning are described as the decision rules that perform well on standard benchmarks but fail to be transferred to more challenging testing scenarios according to [4].

Original: We make amore speciﬁc explanation for easier evaluation.
Revised: We make a more speciﬁc explanation for easier evaluation.

Original: For a task, there are a number of features that shouldbe considered.
Revised: For a task, there are a number of features that should be considered.

Original: Since the training set is limited, perhaps only a subset of features is sufﬁcient to makea decision.
Revised: Since the training set is limited, perhaps only a subset of features is sufﬁcient to make a decision.

Original: Decision rules that rely on an easy-to-learn subset of features are called shortcuts.
Revised: Decision rules that rely on an easy-to-learn subset of features are called shortcuts.

Original: The concept of shortcut has two key points, (1) only use a subset of effective features but perform well ontraining set; (2) easy to learn and affect the learning of other complex but also effective features.
Revised: The concept of shortcut has two key points, (1) only use a subset of effective features but perform well on training set; (2) easy to learn and affect the learning of other complex but also effective features.

Original: Figure 1 provides an intuitive illustration of what shortcuts are and why shortcuts affect the generalization of deep learning models.
Revised: Figure 1 provides an intuitive illustration of what shortcuts are and why shortcuts affect the generalization of deep learning models.

Original: Each sample in Figure 1 has two features (color, shape).
Revised: Each sample in Figure 1 has two features (color, shape).

Original: The trainingset only has red circles and green triangles, while the testing set contains the other two combinations.
Revised: The training set only has red circles and green triangles, while the testing set contains the other two combinations.

Original: For the training set, decision rules are valid whether they are based on color or shape, but both ofthem may fail on the testing set because the training set does not articulate the intent.
Revised: For the training set, decision rules are valid whether they are based on color or shape, but both of them may fail on the testing set because the training set does not articulate the intent.

Original: A task mayrequire the selection of a particular decision rule or a sum-up.
Revised: A task may require the selection of a particular decision rule or a sum-up.

Original: But either way, the diversity of decisionrules is a foundation.
Revised: But either way, the diversity of decision rules is a foundation.

Original: [Figure]
Revised: [Figure]

Original: Inaccessible

Original: [Figure] Some studies expand the difference between decision rules at the feature level [10].
Revised: Some studies expand the difference between decision rules at the feature level [10].

Original: However,differentiated decision rules may still be functionally similar, see Figure 1a.
Revised: However, differentiated decision rules may still be functionally similar, see Figure 1a.

Original: To obtain the functionallydiverse decision rules shown in Figure 1b, we propose to temporarily disable shortcuts.
Revised: To obtain the functionally diverse decision rules shown in Figure 1b, we propose to temporarily disable shortcuts.

Original: It is important to note that shortcuts are disabled temporarily, not forever.
Revised: It is important to note that shortcuts are disabled temporarily, not forever.

Original: We believe that simplicitydoes not mean bad, but more decision rules might help.
Revised: We believe that simplicity does not mean bad, but more decision rules might help.

Original: Our approach is simply providing moreoptions.
Revised: Our approach is simply providing more options.

Original: The integration of decision rules should depend on the task scenarios, and we also providesome suggestions.
Revised: The integration of decision rules should depend on the task scenarios, and we also provide some suggestions.

Original: To address the issue of deep learning methods not learning anything other than shortcuts, we propose
Revised: To address the issue of deep learning methods not learning anything other than shortcuts, we propose

Original: Roadblocks to temporarily disable shortcuts and force the model to learn new knowledge.
Revised: Roadblocks to temporarily disable shortcuts and force the model to learn new knowledge.

Original: The abilityof Roadblocks that disables the shortcuts is achieved by reconstructing datasets under the guidance oftrained models.
Revised: The ability of Roadblocks that disables the shortcuts is achieved by reconstructing datasets under the guidance of trained models.

Original: A reconstructed sample should retain most of the attributes of the original samplewhile disabling the learned attributes.
Revised: A reconstructed sample should retain most of the attributes of the original sample while disabling the learned attributes.

Original: Thus, the model trained with reconstructed samples willnaturally reduce the reliance on shortcuts and learn new knowledge.
Revised: Thus, the model trained with reconstructed samples will naturally reduce the reliance on shortcuts and learn new knowledge.

Original: Enlarging the distance directly [10] at the feature level often only learns the decision rules shown in
Revised: Enlarging the distance directly [10] at the feature level often only learns the decision rules shown in

Original: Figure 1a, but it is difﬁcult to learn functional differences.
Revised: Figure 1a, but it is difﬁcult to learn functional differences.

Original: We believe the reason is the redundancyand weak interpretability of deep learning models.
Revised: We believe the reason is the redundancy and weak interpretability of deep learning models.

Original: There is currently no metric that can measurethe functional differences of the models well.
Revised: There is currently no metric that can measure the functional differences of the models well.

Original: Most previous researches on shortcuts focus on whatfeatures are more likely to be shortcuts, and the corresponding solutions mostly rely on the priorknowledge of shortcuts [5, 8, 18].
Revised: Most previous researches on shortcuts focus on what features are more likely to be shortcuts, and the corresponding solutions mostly rely on the prior knowledge of shortcuts [5, 8, 18].

Original: Priori-based methods require accurate prior knowledge and targeteddesign, which is difﬁcult to accomplish in most scenarios.
Revised: Priori-based methods require accurate prior knowledge and targeted design, which is difﬁcult to accomplish in most scenarios.

Original: Unlike them, Roadblocks do not use anyprior knowledge about shortcuts and can deal with different forms of shortcuts.
Revised: Unlike them, Roadblocks do not use any prior knowledge about shortcuts and can deal with different forms of shortcuts.

Original: In summary, the contributions of this work are mainly three-fold:
Revised: In summary, the contributions of this work are mainly three-fold:

Original: • We propose a new framework to help models learn more diverse knowledge.
Revised: • We propose a new framework to help models learn more diverse knowledge.

Original: Compared with methods based on distance at the feature level, our method can learn brand new knowledge and obtain functionally diverse decision rules.
Revised: Compared with methods based on distance at the feature level, our method can learn brand new knowledge and obtain functionally diverse decision rules.

Original: • We propose a new strategy for disabling shortcuts, which requires no priors and is suitable 
Revised: • We propose a new strategy for disabling shortcuts, which requires no priors and is suitable for a wider range of application scenarios.

Original: for a wider range of application scenarios.• Our method can effectively improve the generalization ability of the model and can reachstate-of-the-arts in the debiasing task.
Revised: • Our method can effectively improve the generalization ability of the model and can reach state-of-the-arts in the debiasing task.

Original: [Figure]
Revised: [Figure]

Original:  Method
Revised:  Method

Original: This section will introduce our strategy to address the shortcut problem, which is temporarilydisabling shortcuts and learning more diverse knowledge.
Revised: This section will introduce our strategy to address the shortcut problem, which is temporarily disabling shortcuts and learning more diverse knowledge.

Original: First, we specify some deﬁnitions and theevaluation system.
Revised: First, we specify some deﬁnitions and the evaluation system.

Original: Then, we introduce our Roadblocks framework.
Revised: Then, we introduce our Roadblocks framework.

Original: Finally, the working process ofthe framework will be introduced.
Revised: Finally, the working process of the framework will be introduced.

Original: 2.1 Shortcut Suppression Task
Revised: 2.1 Shortcut Suppression Task

Original: Shortcuts are described as decision rules that perform well on independently-identical-distribution(i.i.d.) testing sets but fail to generalize to out-of-distribution (o.o.d.) testing sets [4].
Revised: Shortcuts are described as decision rules that perform well on independently-identical-distribution (i.i.d.) testing sets but fail to generalize to out-of-distribution (o.o.d.) testing sets [4].

Original: We refer tothis idea and describe the shortcut more precisely, shortcuts are decision rules that rely on aneasy-to-learn subset of features.
Revised: We refer to this idea and describe the shortcut more precisely, shortcuts are decision rules that rely on an easy-to-learn subset of features.

Original: For classiﬁcation tasks, suppose a distribution D whose feature space can be represented by F .
Revised: For classiﬁcation tasks, suppose a distribution D whose feature space can be represented by F .

Original: Supposing that a training set D train is sampled from D , and there exist a decision rule that basedon F train that can perform well on D train .
Revised: Supposing that a training set D train is sampled from D , and there exist a decision rule that based on F train that can perform well on D train .

Original: F train is a subspace of F .
Revised: F train is a subspace of F .

Original: Such a decision rule has thepotential risk of failing in other datasets sampled from D because it ignores other features in F .
Revised: Such a decision rule has the potential risk of failing in other datasets sampled from D because it ignores other features in F .

Original: For ease of measurement, we refer to the setting of the debiasing task.
Revised: For ease of measurement, we refer to the setting of the debiasing task.

Original: The feature space contains twomain features, F = [ F T , F B ] .
Revised: The feature space contains two main features, F = [ F T , F B ] .

Original: In the training data, the target feature F T corresponds to the groundtruth perfectly; while biased feature F B corresponds to the ground truth with a large probability.
Revised: In the training data, the target feature F T corresponds to the ground truth perfectly; while biased feature F B corresponds to the ground truth with a large probability.

Original: Nonetheless, there is still a possibility that F train is closer to F B because F B is simpler, whichmeans that the model may not have learned F T as we expected.
Revised: Nonetheless, there is still a possibility that F train is closer to F B because F B is simpler, which means that the model may not have learned F T as we expected.

Original: To verify the learning of F T in thetesting data, F T still corresponds to the ground truth perfectly, and F B is no longer helpful.
Revised: To verify the learning of F T in the testing data, F T still corresponds to the ground truth perfectly, and F B is no longer helpful.

Original: 2.2 Framework of Roadblocks
Revised: 2.2 Framework of Roadblocks

Original: Assuming that existing models use shortcuts, a strategy to ﬁnd more reasonable decision rules isto use the idea of Roadblocks that forces the new model not to use the same decision rule as anyprevious model.
Revised: Assuming that existing models use shortcuts, a strategy to ﬁnd more reasonable decision rules is to use the idea of Roadblocks that forces the new model not to use the same decision rule as any previous model.

Original: Figure 2 illustrates the framework of our approach.
Revised: Figure 2 illustrates the framework of our approach.

Original: The overall model is composed of three parts.
Revised: The overall model is composed of three parts.

Original: Blocked models are trained models that perform well on the i.i.d.
Revised: Blocked models are trained models that perform well on the i.i.d.

Original: test dataset.
Revised: test dataset.

Original: The autoencoder isused to modify the original input, thereby limiting the performance of blocked models.
Revised: The autoencoder is used to modify the original input, thereby limiting the performance of blocked models.

Original: Once theblocked models perform poorly on the modiﬁed input, the explorer models trained with the modiﬁedinput will ﬁnd new ways to achieve competitive performance.
Revised: Once the blocked models perform poorly on the modiﬁed input, the explorer models trained with the modiﬁed input will ﬁnd new ways to achieve competitive performance.

Original: None of the three sub-models need aspeciﬁc structure, as long as they have sufﬁcient representation capabilities.
Revised: None of the three sub-models need a speciﬁc structure, as long as they have sufﬁcient representation capabilities.

Original: Let AE, BM, EM denote autoencoder, blocked model, explorer model, respectively.
Revised: Let AE, BM, EM denote autoencoder, blocked model, explorer model, respectively.

Original: BM is a pretrained model with good performance, which is ﬁxed during the training process.
Revised: BM is a pretrained model with good performance, which is ﬁxed during the training process.

Original: AE receives theraw input I raw and modiﬁes it into blocked input I rec .
Revised: AE receives the raw input I raw and modiﬁes it into blocked input I rec .

Original: The mean square error  is used to measure the similarity between them,
Revised: The mean square error (MSE) is used to measure the similarity between them,

Original: [Equation]
Revised: [Equation]

Original: The reconstructed input needs to meet two requirements, (1) it is difﬁcult to distinguish with theexisting decision rules, and (2) it can be correctly distinguished.
Revised: The reconstructed input needs to meet two requirements, (1) it is difﬁcult to distinguish with the existing decision rules, and (2) it can be correctly distinguished.

Original: I rec will ﬁrst be used as the input of BM.
Revised: I rec will ﬁrst be used as the input of BM.

Original: The MSE of the prediction BM ( I rec ) and the pseudo labellabel ep will be used as the roadblock loss,
Revised: The MSE of the prediction BM ( I rec ) and the pseudo label label ep will be used as the roadblock loss,

Original: [Equation]
Revised: [Equation]

Original: The pseudo label label ep is set to consider all possible categories with equal probability, which islabel ep =
Revised: The pseudo label label ep is set to consider all possible categories with equal probability, which is label ep =

Original: [ 1 n , 1 n , ..., 1 n ]) , where n is the number of categories.
Revised: [ 1 n , 1 n , ..., 1 n ]) , where n is the number of categories.

Original: In this way, the purpose of AE is togenerate I rec to confuse BM.
Revised: In this way, the purpose of AE is to generate I rec to confuse BM.

Original: I rec will also be used as the input of EM.
Revised: I rec will also be used as the input of EM.

Original: In order for the EM to achieve better classiﬁcationperformance, the accuracy loss between the prediction EM ( I rec ) and the ground-truth label iscalculated for training,
Revised: In order for the EM to achieve better classiﬁcation performance, the accuracy loss between the prediction EM ( I rec ) and the ground-truth label is calculated for training,

Original: [Equation]
Revised: [Equation]

Original: 2.3 Working Process
Revised: 2.3 Working Process

Original: Core training process BM is a pre-trained model, and it does not participate in the training processof the proposed framework.
Revised: Core training process BM is a pre-trained model, and it does not participate in the training process of the proposed framework.

Original: Both AE and EM need to be trained.
Revised: Both AE and EM need to be trained.

Original: We alternately train them for amore stable training process.
Revised: We alternately train them for a more stable training process.

Original: A typical training cycle consists of two parts, (1) AE is trained withroadblock loss and reconstruction loss; (2) EM is trained under the guidance of accuracy loss,[Equation]
Revised: A typical training cycle consists of two parts, (1) AE is trained with roadblock loss and reconstruction loss; (2) EM is trained under the guidance of accuracy loss,

Original: [Equation]
Revised: [Equation]

Original: Here, θ AE and θ EM denote the parameters of AE and EM respectively, and λ =
Revised: Here, θ AE and θ EM denote the parameters of AE and EM respectively, and λ =

Original: [ λ rb , λ rec , λ ac ] arethe corresponding loss weights.
Revised: [ λ rb , λ rec , λ ac ] are the corresponding loss weights.

Original: Multi-model BM can be any model that performs well on training set.
Revised: Multi-model BM can be any model that performs well on training set.

Original: In addition to vanilla models,a EM generated by Roadblocks method can also be used as BM to train a new EM.
Revised: In addition to vanilla models, a EM generated by Roadblocks method can also be used as BM to train a new EM.

Original: BM can also bea set of models BM =
Revised: BM can also be a set of models BM =

Original: [ BM 1 , BM 2 , ..., BM n ] .
Revised: [ BM 1 , BM 2 , ..., BM n ] .

Original: Correspondingly, the roadblock loss needs to beadjusted to
Revised: Correspondingly, the roadblock loss needs to be adjusted to

Original: [Equation]
Revised: [Equation]

Original: We suggest adding the well-trained EM to the set of BM and then training the new EM.
Revised: We suggest adding the well-trained EM to the set of BM and then training the new EM.

Original: When thenew EM fails to perform well on the training set, it means that it is difﬁcult to learn more effectivefeatures, at which point the loop terminates.
Revised: When the new EM fails to perform well on the training set, it means that it is difﬁcult to learn more effective features, at which point the loop terminates.

Original: The experiments suggest that in most cases, two modelsare sufﬁcient.
Revised: The experiments suggest that in most cases, two models are sufﬁcient.

Original: Therefore, to reduce the computational cost, we usually use two models.
Revised: Therefore, to reduce the computational cost, we usually use two models.

Original: Sum-up The proposed framework contains multiple sub-models that learn different effective featuresrespectively.
Revised: Sum-up The proposed framework contains multiple sub-models that learn different effective features respectively.

Original: We do not recommend discarding any features directly, as the ease of learning and theeffectiveness of features are not necessarily related.
Revised: We do not recommend discarding any features directly, as the ease of learning and the effectiveness of features are not necessarily related.

Original: We recommend designing according to speciﬁcscenarios.
Revised: We recommend designing according to speciﬁc scenarios.

Original: Without priors on the test scene, we recommend using the feature fusion strategy becauseit is more likely to have good generalization.
Revised: Without priors on the test scene, we recommend using the feature fusion strategy because it is more likely to have good generalization.

Original: Another idea is to vote, and o.o.d.
Revised: Another idea is to vote, and o.o.d.

Original: samples can bedetected if there is a signiﬁcant difference in the votes of the submodels.
Revised: samples can be detected if there is a signiﬁcant difference in the votes of the submodels.

Original: For some tasks with priors,the appropriate model can be directly selected.
Revised: For some tasks with priors, the appropriate model can be directly selected.

Original: For example, for debiasing tasks, a reasonable ideais to choose the model that performs better on the training set because the biased features are lessmatched with the ground truth.
Revised: For example, for debiasing tasks, a reasonable idea is to choose the model that performs better on the training set because the biased features are less matched with the ground truth.

Original:  Experiments
Revised:  Experiments

Original: To verify whether functionally different decision rules are learned, we choose datasets with multipleattribute labels for experiments.
Revised: To verify whether functionally different decision rules are learned, we choose datasets with multiple attribute labels for experiments.

Original: The experimental setup refers to the design of the debiasing task[15].
Revised: The experimental setup refers to the design of the debiasing task [15].

Original: Each individual attribute in the training set corresponds well to the ground truth, while onlyone attribute matches the ground truth one-to-one in the testing set.
Revised: Each individual attribute in the training set corresponds well to the ground truth, while only one attribute matches the ground truth one-to-one in the testing set.

Original: Taking Figure 1 as an example,
Revised: Taking Figure 1 as an example,

Original: [Table] [Table]
Revised: [Table] [Table]

Original: the training set is mainly red circles and green triangles, and the four combinations have the sameproportion in the testing set.
Revised: the training set is mainly red circles and green triangles, and the four combinations have the same proportion in the testing set.

Original: We use the synthetic dataset CMNIST [15], which adds a secondattribute by coloring MNIST [14].
Revised: We use the synthetic dataset CMNIST [15], which adds a second attribute by coloring MNIST [14].

Original: We also use real-world datasets CelebA [20], and BAR [23]that were validated to have shortcuts to test the practicality of our method.
Revised: We also use real-world datasets CelebA [20], and BAR [23] that were validated to have shortcuts to test the practicality of our method.

Original: Details of datasetsand implementation are described in the appendix.
Revised: Details of datasets and implementation are described in the appendix.

Original: Extensive experiments have been conductedto demonstrate the following effects of Roadblocks: 1) greatly enhance the ability to explore newknowledge, 2) signiﬁcantly improve the generalization of deep learning models.
Revised: Extensive experiments have been conducted to demonstrate the following effects of Roadblocks: 1) greatly enhance the ability to explore new knowledge, 2) signiﬁcantly improve the generalization of deep learning models.

Original: 3.1 Exploration of Knowledge
Revised: 3.1 Exploration of Knowledge

Original: To verify whether Roadblocks can reduce models’ reliance on shortcuts, we compared the performanceof the conventionally trained CNNs (Vanilla) and some representative debiasing methods and our
Revised: To verify whether Roadblocks can reduce models’ reliance on shortcuts, we compared the performance of the conventionally trained CNNs (Vanilla) and some representative debiasing methods and our

Original: Roadblocks method.
Revised: Roadblocks method.

Original: We ﬁrst test on a controlled datasets CMNIST to facilitate analysis.
Revised: We ﬁrst test on a controlled datasets CMNIST to facilitate analysis.

Original: Theexperimental results are shown in Table 1.
Revised: The experimental results are shown in Table 1.

Original: For CMNIST, each sample has two key features, color, and digit.
Revised: For CMNIST, each sample has two key features, color, and digit.

Original: Although the colors are slightlyless matched with ground truth, deep learning models tend to learn colors.
Revised: Although the colors are slightly less matched with ground truth, deep learning models tend to learn colors.

Original: Our method can help deeplearning models learn more decision rules that utilize digit features.
Revised: Our method can help deep learning models learn more decision rules that utilize digit features.

Original: Models also often rely on shortcuts to complete their tasks in real-world scenarios.
Revised: Models also often rely on shortcuts to complete their tasks in real-world scenarios.

Original: We conductedexperiments on the face attribute dataset CelebA [20] and the action recognition dataset BAR [23].
Revised: We conducted experiments on the face attribute dataset CelebA [20] and the action recognition dataset BAR [23].

Original: For real-world datasets, we keep its native ratio to maintain the authenticity during the trainingprocess.
Revised: For real-world datasets, we keep its native ratio to maintain the authenticity during the training process.

Original: And then we test whether the model learned more knowledge through testing sets of differentdistributions.
Revised: And then we test whether the model learned more knowledge through testing sets of different distributions.

Original: For CelebA, following the experimental setup of LFF [23], we trained models toclassify ‘HeavyMakeup’.
Revised: For CelebA, following the experimental setup of LFF [23], we trained models to classify ‘HeavyMakeup’.

Original: Then we select the same number of samples from all possible combinationsof [Gender, HeavyMakeup] to form the unbiased testing set and minority combinations to form thebias-conﬂicting testing set.
Revised: Then we select the same number of samples from all possible combinations of [Gender, HeavyMakeup] to form the unbiased testing set and minority combinations to form the bias-conﬂicting testing set.

Original: BAR contains six actions, and the backgrounds of the training and testingsets are signiﬁcantly different.
Revised: BAR contains six actions, and the backgrounds of the training and testing sets are signiﬁcantly different.

Original: The testing set has only bias-conﬂict samples.
Revised: The testing set has only bias-conﬂict samples.

Original: The results are shownin Table 2.
Revised: The results are shown in Table 2.

Original: It can be seen that our method has obvious performance improvement compared to thestate-of-the-art methods.
Revised: It can be seen that our method has obvious performance improvement compared to the state-of-the-art methods.

Original: In particular, for the bias-conﬂict dataset which is more difﬁcult, our methodachieves more than 10% improvement.[Figure]
Revised: In particular, for the bias-conﬂict dataset which is more difﬁcult, our method achieves more than 10% improvement.

Original: [Figure]
Revised: [Figure]

Original: 3.2 Disabling shortcuts
Revised: 3.2 Disabling Shortcuts

Original: [Figure]

Original: The above experiments veriﬁed the effectiveness of our Roadblocks framework in learning functionally diverse decision rules.
Revised: The above experiments veriﬁed the effectiveness of our Roadblocks framework in learning functionally diverse decision rules.

Original: Next, it is necessary to conﬁrm whether the effective mechanism of theproposed method is consistent with the original intention of the design.
Revised: Next, it is necessary to conﬁrm whether the effective mechanism of the proposed method is consistent with the original intention of the design.

Original: Speciﬁcally, it is necessaryto verify that some information in the reconstructed image is indeed corrupted so that the existingdecision rules cannot make correct predictions.
Revised: Speciﬁcally, it is necessary to verify that some information in the reconstructed image is indeed corrupted so that the existing decision rules cannot make correct predictions.

Original: Thus, we visualized reconstructed images, and theresult is shown in Figure 3.
Revised: Thus, we visualized reconstructed images, and the result is shown in Figure 3.

Original: It can be observed that for CMNIST, our method can implement thedisabling of the shortcut (color) using different strategies, such as using mixed colors or generatingnew colors.
Revised: It can be observed that for CMNIST, our method can implement the disabling of the shortcut (color) by repainting digits.

Original: Both of these modiﬁcations make it difﬁcult for themodel to use color for classiﬁcation. Meanwhile, the shape of the numbers has not changed.
Revised: For different digits, the strategy of repainting is consistent, which shows that only the color attribute is disabled in the painting stage, while the number attribute is retained.

Original: Therefore, it is still feasible to use thesesamples to learn numbers, which is also well illustrated in Table 1.
Revised: Therefore, it is still feasible to use these samples to learn numbers, which is also well illustrated in Table 1.

Original: For real-world datasets, similarmodiﬁcation methods are challenging to achieve, and our approach mainly implements the destructionby adding noise.
Revised: For real-world datasets, similar modiﬁcation methods are challenging to achieve, and our approach mainly implements the destruction by adding noise.

Original: Forexample for CelebA, AE tries to blur the gender attribute by some blurring(mainly focused on the face contour).
Revised: For CelebA, AE tries to blur the gender attribute by some blurring (mainly focused on the face contour).

Original: For BAR, the blurring of AE mainly revolves around theenvironment.
Revised: For BAR, the blurring of AE mainly revolves around the environment.

Original: For the convenience of observation, we show the difference map of the original imageand the reconstructed image, which is ﬁve times the absolute value of the difference of each pixelvalue between the two.
Revised: For the convenience of observation, we show the difference map of the original image and the reconstructed image, which is ﬁve times the absolute value of the difference of each pixel value between the two.

Original: We also used t-SNE [30] to visualize the features extracted by the vanilla model and our approach,and the result is shown in Figure 4.
Revised: We also used t-SNE [30] to visualize the features extracted by the vanilla model and our approach, and the result is shown in Figure 4.

Original: Each colored number represents a sample.
Revised: Each colored number represents a sample.

Original: It is evident thatthe vanilla model does an excellent job of partitioning the dataset according to color.
Revised: It is evident that the vanilla model does an excellent job of partitioning the dataset according to color.

Original: However, acloser look reveals that samples of the same color, regardless of the digit, are in the same cluster.
Revised: However, a closer look reveals that samples of the same color, regardless of the digit, are in the same cluster.

Original: Incontrast, the model trained by our approach can distinguish digits and have certain invariance to color.
Revised: In contrast, the model trained by our approach can distinguish digits and have certain invariance to color.

Original: Distinguishing digits is more complex than color, so getting sharp boundaries is more complicated.
Revised: Distinguishing digits is more complex than color, so getting sharp boundaries is more complicated.

Original: To visually demonstrate the model’s ability to explore new knowledge, we use Grad-CAM [26] tovisualize the focus of models on the BAR dataset, and saliency maps are shown in Figure 5.
Revised: To visually demonstrate the model’s ability to explore new knowledge, we use Grad-CAM [26] to visualize the focus of models on the BAR dataset, and saliency maps are shown in Figure 5.

Original: In thisexperiment, we use ResNet-18 as the backbone, and the last fully connected layer is modiﬁed to alinear layer (512, 10) to match the number of categories.
Revised: In this experiment, we use ResNet-18 as the backbone, and the last fully connected layer is modiﬁed to a linear layer (512, 10) to match the number of categories.

Original: And the last convolutional layer is usedas the target layer of Grad-CAM.
Revised: And the last convolutional layer is used as the target layer of Grad-CAM.

Original: For CMNIST, it is difﬁcult to tell the difference from the saliencymap because the colors and shapes correspond to the same regions.
Revised: For CMNIST, it is difﬁcult to tell the difference from the saliency map because the colors and shapes correspond to the same regions.

Original: This is also the reason why wedo not disable shortcuts using saliency map based methods.
Revised: This is also the reason why we do not disable shortcuts using saliency map based methods.

Original: For CelebA and BAR, it can be seenthat the attention regions of our approach are signiﬁcantly different from that of the vanilla model.
Revised: For CelebA and BAR, it can be seen that the attention regions of our approach are signiﬁcantly different from that of the vanilla model.

Original: For correctly classiﬁed samples, our approach can ﬁnd more effective information.
Revised: For correctly classiﬁed samples, our approach can ﬁnd more effective information.

Original: Grad-CAM alsoprovides some explanation for samples that vanilla misclassiﬁes, but our method classiﬁes correctly.
Revised: Grad-CAM also provides some explanation for samples that vanilla misclassiﬁes, but our method classiﬁes correctly.

Original: For example, for samples of mountain climbing, the EM of our approach focuses on the climber, andthe vanilla model focuses on the mountain.
Revised: For example, for samples of mountain climbing, the EM of our approach focuses on the climber, and the vanilla model focuses on the mountain.

Original: Both of these features are important, and our approachhas better generalization ability because it utilizes more effective features.
Revised: Both of these features are important, and our approach has better generalization ability because it utilizes more effective features.

Original: The samples of CMNIST 
Revised: The samples of CMNIST and CelebA were randomly selected, and we selected more representative samples for BAR. There

Original: [Figure]
Revised: [Figure]

Original: and CelebA were randomly selected, and we selected more representative samples for BAR.

Original: Thereare more examples of samples in the appendix to facilitate more analysis and discussion, while alsoensuring generality.
Revised: are more examples of samples in the appendix to facilitate more analysis and discussion, while also ensuring generality.

Original: Grad-CAM is an excellent interpretability method, but it can only show differences in models at aspatial scale.
Revised: Grad-CAM is an excellent interpretability method, but it can only show differences in models at a spatial scale.

Original: Therefore, for datasets such as CMNIST, where different features overlap a lot in spatialscales, saliency maps cannot well reﬂect the differences between models.
Revised: Therefore, for datasets such as CMNIST, where different features overlap a lot in spatial scales, saliency maps cannot well reﬂect the differences between models.

Original: The reconstruction results and saliency maps visually demonstrate that our approach does learn moreeffective features following our design intent.
Revised: The reconstruction results and saliency maps visually demonstrate that our approach does learn more effective features following our design intent.

Original: These methods still have some limitations.
Revised: These methods still have some limitations.

Original: To verifythe ability of our method to improve model diversity as fully as possible, we further quantify themodel diversity.
Revised: To verify the ability of our method to improve model diversity as fully as possible, we further quantify the model diversity.

Original: 3.3 Model Diversity
Revised: 3.3 Model Diversity

Original: Our core purpose is to learn more knowledge (i.e., effective features).
Revised: Our core purpose is to learn more knowledge (i.e., effective features).

Original: Therefore, it is necessaryto choose appropriate metrics to measure the similarity between models.
Revised: Therefore, it is necessary to choose appropriate metrics to measure the similarity between models.

Original: We applied SVCCA [24]to measure the similarity of the models.
Revised: We applied SVCCA [24] to measure the similarity of the models.

Original: SVCCA is a tool for comparing two representations in away that is invariant to afﬁne transform.
Revised: SVCCA is a tool for comparing two representations in a way that is invariant to afﬁne transform.

Original: It receives the feature maps of middle layers and calculatesthe similarity between each pair of feature maps.
Revised: It receives the feature maps of middle layers and calculates the similarity between each pair of feature maps.

Original: Figure 6 shows some SVCCA results betweenthe models.
Revised: Figure 6 shows some SVCCA results between the models.

Original: Figure 6a and Figure 6b are calculated on CMNIST (5pct), Figure 6c and Figure 6dare calculated on BAR.
Revised: Figure 6a and Figure 6b are calculated on CMNIST (5pct), Figure 6c and Figure 6d are calculated on BAR.

Original: CelebA is too large to be calculated, so we did not perform correspondingexperiments.
Revised: CelebA is too large to be calculated, so we did not perform corresponding experiments.

Original: Vanilla
Revised: Vanilla

Original: V a n ill a 
Revised: V a n ill a (a) CMNIST, V-V

Original: [Figure]
Revised: Ours V a n ill a (b) CMNIST, V-O

Original: Vanilla
Revised: Vanilla

Original: V a n ill a [Figure]
Revised: V a n ill a (c) BAR, V-V

Original: Ours
Revised: Ours

Original: V a n ill a [Figure]
Revised: V a n ill a (d) BAR, V-O

Original: For the CMNIST dataset, the diagonal of the SVCCA matrix of the vanilla model is bright, indicatinga high similarity between the same layers.
Revised: For the CMNIST dataset, the diagonal of the SVCCA matrix of the vanilla model is bright, indicating a high similarity between the same layers.

Original: While the similarity between our model and the vanillamodel is slight, there are only a few brighter regions between several lower layers.
Revised: While the similarity between our model and the vanilla model is slight, there are only a few brighter regions between several lower layers.

Original: The comparison ofthe two shows that the model trained by our method is signiﬁcantly different from the vanilla modelat the feature level.
Revised: The comparison of the two shows that the model trained by our method is signiﬁcantly different from the vanilla model at the feature level.

Original: Therefore, our method helps to improve the diversity of models.
Revised: Therefore, our method helps to improve the diversity of models.

Original: It can also beobserved that some adjacent layers also have high similarities.
Revised: It can also be observed that some adjacent layers also have high similarities.

Original: The BAR dataset is more complexthan the CMNIST dataset.
Revised: The BAR dataset is more complex than the CMNIST dataset.

Original: Therefore, the similarity between adjacent layers in the vanilla model islower.
Revised: Therefore, the similarity between adjacent layers in the vanilla model is lower.

Original: Some layers of our model and the corresponding layers vanilla model are similar, which wethink may correspond to some complex semantic features.
Revised: Some layers of our model and the corresponding layers vanilla model are similar, which we think may correspond to some complex semantic features.

Original: 3.4 Sensitivity to Hyperparameters

Original: Our method requires training an autoencoder and a classiﬁer model, involving three loss functions.

Original: Inthis section, we discuss the sensitivity to hyperparameters of our approach.

Original: We test the performanceson debiasing tasks under different loss weights ( λ ).

Original: We set λ ac as 1, and tested different λ rb andλ rec .

Original: The experimental results are shown in the appendix.

Original: It can be seen that our method achievescompetitive performance for an extensive range of λ .

Original: To make the comparison meaningful, here weshow the result of EM, regardless of how it performs during the training phase.

Original: When the weight ofthe roadblock loss is too large, and the weight of the reconstruction loss is too small, the informationrequired for classiﬁcation is lacking in the modiﬁed input, resulting in poor classiﬁcation accuracy.

Original: Itis not difﬁcult to understand that our approach is not very sensitive to the change of hyperparameters.

Original: Only one more autoencoder AE needs to be trained, and the objectives of AE and EM are differentbut not conﬂicting.

Original: Thus, simple alternating training can bring both AE and EM to convergence.

Original:  Related Work and Discussion 
Revised:  Related Work and Discussion 4.1 Related Work

Original: 4.1 Related Work

Original: Shortcut learning .
Revised: Shortcut learning .

Original: Geirhos et al.
Revised: Geirhos et al.

Original: [4] provided a comprehensive summary of shortcuts and deﬁnedthem as decision rules that perform poorly on the o.o.d.
Revised: [4] provided a comprehensive summary of shortcuts and deﬁned them as decision rules that perform poorly on the o.o.d.

Original: testing set.
Revised: testing set.

Original: A considerable amount of researchhas been devoted to analysing the formation of shortcuts, providing many valuable conclusions[27, 22].
Revised: A considerable amount of research has been devoted to analysing the formation of shortcuts, providing many valuable conclusions [27, 22].

Original: Deep learning models is found to have the tendency to use textures rather than shapesto make predictions [5], which has inspired a new round of thinking among researchers on themechanism of deep learning.
Revised: Deep learning models is found to have the tendency to use textures rather than shapes to make predictions [5], which has inspired a new round of thinking among researchers on the mechanism of deep learning.

Original: Katherine and Andrew [9] explored the conditions under which featuresare more likely to be shortcuts, which is very helpful to deepen our understanding of shortcuts.
Revised: Katherine and Andrew [9] explored the conditions under which features are more likely to be shortcuts, which is very helpful to deepen our understanding of shortcuts.

Original: Inaddition, various data augmentation methods are used to drive models to learn features beyond knownshortcuts.
Revised: In addition, various data augmentation methods are used to drive models to learn features beyond known shortcuts.

Original: Some researches [5, 8, 18] assumed that the texture feature is the only shortcut, and thedesigned data augmentation techniques effectively reduce the model’s reliance on texture.
Revised: Some researches [5, 8, 18] assumed that the texture feature is the only shortcut, and the designed data augmentation techniques effectively reduce the model’s reliance on texture.

Original: There arealso ways to exclude shortcuts by introducing a prior on the difﬁculty of the target task [3].
Revised: There are also ways to exclude shortcuts by introducing a prior on the difﬁculty of the target task [3].

Original: However,research on suppressing shortcut without relying on prior knowledge is still basically an unknownarea to explore.
Revised: However, research on suppressing shortcut without relying on prior knowledge is still basically an unknown area to explore.

Original: Fairness and debiasing .
Revised: Fairness and debiasing .

Original: The debiasing task can be understood as solving the problem that the modelhas learned a certain bias attribute in the data set, but failed to learn the relatively difﬁcult targetattribute.
Revised: The debiasing task can be understood as solving the problem that the model has learned a certain bias attribute in the data set, but failed to learn the relatively difﬁcult target attribute.

Original: Some researchers have explored the preferences of deep learning methods themselves and dealt with them accordingly, such as the tendency to use textures rather than shapes [5] and thetendency to learn low-frequency features in images [2].
Revised: Some researchers have explored the preferences of deep learning methods themselves and dealt with them accordingly, such as the tendency to use textures rather than shapes [5] and the tendency to learn low-frequency features in images [2].

Original: Besides, several existing approaches aimedat mitigating the bias also assume certain bias types [12, 31, 17, 29, 25].
Revised: Besides, several existing approaches aimed at mitigating the bias also assume certain bias types [12, 31, 17, 29, 25].

Original: Instead of deﬁning speciﬁctypes of bias, some approaches [23, 15, 19, 16] rely on the straightforward assumption that simplefeatures are biased features.
Revised: Instead of deﬁning speciﬁc types of bias, some approaches [23, 15, 19, 16] rely on the straightforward assumption that simple features are biased features.

Original: A typical example is Learn from Failure .
Revised: A typical example is Learn from Failure .

Original: The main idea is twofold; (a)intentionally train the ﬁrst network to be biased by repeatedly amplifying its “prejudice”, and (b)debias the training of the second network by focusing on samples that go against the prejudice of thebiased network in (a).
Revised: The main idea is twofold; (a) intentionally train the ﬁrst network to be biased by repeatedly amplifying its “prejudice”, and (b) debias the training of the second network by focusing on samples that go against the prejudice of the biased network in (a).

Original: Unlike them, our approach does not use biased priors, but solves the debiasingproblem by learning more knowledge.
Revised: Unlike them, our approach does not use biased priors, but solves the debiasing problem by learning more knowledge.

Original: Compared with other debiasing methods, it has a wider rangeof application scenarios.
Revised: Compared with other debiasing methods, it has a wider range of application scenarios.

Original: Adversarial training The proposed roadblocks share some similarities with adversarial training.
Revised: Adversarial training The proposed roadblocks share some similarities with adversarial training.

Original: Adversarial examples obtained by adding small designed perturbations can make the model go wrong[28].
Revised: Adversarial examples obtained by adding small designed perturbations can make the model go wrong [28].

Original: Adversarial training [11] can effectively improve the adversarial robustness of the model.
Revised: Adversarial training [11] can effectively improve the adversarial robustness of the model.

Original: Ourapproach and adversarial training clearly differ in purpose, which also introduces design differences.
Revised: Our approach and adversarial training clearly differ in purpose, which also introduces design differences.

Original: Adversarial examples only require the model to predict incorrectly, whereas our method requires themodel to be completely confused.
Revised: Adversarial examples only require the model to predict incorrectly, whereas our method requires the model to be completely confused.

Original: Therefore, by design, the false labels we use are all categories,including the true category, with equal probability.
Revised: Therefore, by design, the false labels we use are all categories, including the true category, with equal probability.

Original: We also conduct experiments using adversarialtraining, and the results are shown in the appendix.
Revised: We also conduct experiments using adversarial training, and the results are shown in the appendix.

Original: 4.2 Discussion
Revised: 4.2 Discussion

Original: Functionally identical features may have different representations.
Revised: Functionally identical features may have different representations.

Original: Increasing the diversity offeatures, in other words, learning more knowledge, is a natural idea, but its implementation is notas simple.
Revised: Increasing the diversity of features, in other words, learning more knowledge, is a natural idea, but its implementation is not as simple.

Original: An intuitive way to learn different features is to expand the distance between the twomodels at the feature level.
Revised: An intuitive way to learn different features is to expand the distance between the two models at the feature level.

Original: We refer to the framework of negative correlation learning and expand thedistance between models at the feature level as much as possible on the premise of ensuring consistentprediction results.
Revised: We refer to the framework of negative correlation learning and expand the distance between models at the feature level as much as possible on the premise of ensuring consistent prediction results.

Original: Experimental results (see  appendix) show that it is difﬁcult to learn decision rulesother than shortcuts through such a learning paradigm.
Revised: Experimental results (see the appendix) show that it is difﬁcult to learn decision rules other than shortcuts through such a learning paradigm.

Original: We believe that the reason is that the excellentrepresentational ability of the deep learning model allows it to represent functionally identical featuresin different ways.
Revised: We believe that the reason is that the excellent representational ability of the deep learning model allows it to represent functionally identical features in different ways.

Original: Therefore, a simpler and more effective approach is our Roadblocks, which forcesthe model to learn new features by temporarily disabling the already learned features by modifyingthe input.
Revised: Therefore, a simpler and more effective approach is our Roadblocks, which forces the model to learn new features by temporarily disabling the already learned features by modifying the input.

Original: Rather than some distance between features, we remind that functional differences betweenfeatures may be more important.
Revised: Rather than some distance between features, we remind that functional differences between features may be more important.

Original: From comprehensiveness to exactness.
Revised: From comprehensiveness to exactness.

Original: Our method is designed for general image classiﬁcationtasks.
Revised: Our method is designed for general image classiﬁcation tasks.

Original: The priors about the features and the distribution difference between training and testing setsare unknown.
Revised: The priors about the features and the distribution difference between training and testing sets are unknown.

Original: On the one hand, comprehensiveness (i.e., maximizing the use of effective features inthe accessible data) is a reasonable goal since such models are the most robust against complex testscenarios.
Revised: On the one hand, comprehensiveness (i.e., maximizing the use of effective features in the accessible data) is a reasonable goal since such models are the most robust against complex test scenarios.

Original: For comprehensiveness, our approach allows the model to temporarily disable shortcuts andlearn new effective features as much as possible.
Revised: For comprehensiveness, our approach allows the model to temporarily disable shortcuts and learn new effective features as much as possible.

Original: Its effectiveness and beneﬁts have been describedabove.
Revised: Its effectiveness and beneﬁts have been described above.

Original: On the other hand, exactness (i.e., only using some speciﬁc features) is a more realistic goalfor many tasks.
Revised: On the other hand, exactness (i.e., only using some speciﬁc features) is a more realistic goal for many tasks.

Original: Our approach also provides a new implementation for feature screening.
Revised: Our approach also provides a new implementation for feature screening.

Original: By ﬁrsttraining a model to learn features that are not required for the task and then restricting the use of suchfeatures, it will be easier for the model to follow the designer’s intent.
Revised: By ﬁrst training a model to learn features that are not required for the task and then restricting the use of such features, it will be easier for the model to follow the designer’s intent.

Original: Achieving the requirement forexactness at the lowest cost will be a key point for the designer.
Revised: Achieving the requirement for exactness at the lowest cost will be a key point for the designer.

Original:  Conclusion
Revised:  Conclusion

Original: In this work, we develop a novel framework to prevent deep learning models from merely usingshortcuts (simple decision rules) while ignoring other effective features.
Revised: In this work, we develop a novel framework to prevent deep learning models from merely using shortcuts (simple decision rules) while ignoring other effective features.

Original: Experimental results onmulti-attribute datasets in complex testing scenarios demonstrate that our Roadblocks have a strongcapacity of reducing reliance on shortcuts and learning new knowledge, which is very difﬁcult forconventional deep learning methods and debiasing models.
Revised: Experimental results on multi-attribute datasets in complex testing scenarios demonstrate that our Roadblocks have a strong capacity of reducing reliance on shortcuts and learning new knowledge, which is very difﬁcult for conventional deep learning methods and debiasing models.

Original: Extensive experiments also show that
Revised: Extensive experiments also show that

Original: Roadblocks achieve excellent performance in terms of extracting new knowledge and improvingmodel diversity.
Revised: Roadblocks achieve excellent performance in terms of extracting new knowledge and improving model diversity.

Original: We expect our Roadblocks may help shed light on the nature of shortcut learning inneural networks, and provide a 
Revised: We expect our Roadblocks may help shed light on the nature of shortcut learning in neural networks, and provide a feasible paradigm for training models without depending on shortcuts.  Acknowledgments

Revised: This work was supported by the Anhui Provincial Natural Science Foundation under Grant 2108085UD12, and in part by the National Natural Science Foundation of China under Grants

Revised: U19B2044 and 61836011.

Revised: We acknowledge the support of GPU cluster built by MCC Lab of

Revised: Information Science and Technology Institution, USTC.

Revised: A Appendix A.1 Implementation Details

Revised: Our code is mainly based on pytorch and will be available at https://github.com/badcode-iip/

Revised: Roadblocks .

Revised: The datasets used in the paper are all public datasets.

Revised: There are many versions of

Revised: CMNIST, the one we use is the version as LFF [23].

Revised: It contains identically distributed training and validation sets, and an unbiased testing set.

Revised: We did not use any preprocessing for CMNIST.

Revised: The details and download methods of CelebA [20] and BAR [23] can be found in the corresponding papers.

Revised: We preprocessed CelebA and BAR (resize, normalize) for easier training, we also applied horizontal ﬂipping for augmentation.

Revised: The implementation of t-SNE is from scikit-learn, and the implementations of Grad-CAM and SVCCA are all from open source code.

Revised: For Table 1, we use 3-layer MLPs, the number of neurons are (3*28*28, 100, 100, 32, 10).

Revised: For the rest of the experiments, we use ResNet-18 as the classiﬁer, and the pre-training parameters provided by pytorch only in experiments on CelebA and BAR.

Revised: The autoencoder is consist of a encoder (3 Conv2d layer) and a correspond decoder.

Revised: We use Adam as the optimizer, learning rate is 1e-3 and alpha is 1e-6.

Revised: The max epoch is 100.

Revised: The batch size is 64 for CMNIST and 16 for CelebA and BAR.

Revised: [ λ ac , λ rb , λ rec ] are set to [1, 1, 10] for CelebA and [1, 0.01, 10] for CMNIST and BAR.

Revised: A.2 Functionally identical features may have different representations.

Revised: We use negative correlation learning to train models with feature-level differences, hoping to learn different decision rules.

Revised: In order to show the results more intuitively, we divide the testing set into independently-identical-distribution (i.i.d.) part and out-of-distribution (o.o.d.)

Revised: part.

Revised: The results are shown in Table 3.

Revised: It can be seen that none of them can classify o.o.d.

Revised: samples correctly, which means that none of them can learn features other than color.

Revised: [Table]

Revised: A.3 What if autoencoder is not strong enough?

Revised: The reconstruction capability of AE has a direct impact on the setting of Roadblocks.

Revised: In this section we use 3-layer MLPs to build AE.

Revised: By adjusting the reconstruction ability and the weight of reconstruction, we conduct further analysis on the behavior of the model.

Revised: While keeping other hyperparameters unchanged, we modiﬁed the number of neurons in the code layer of AE to limit its reconstruction capability, then we repeated the previous experiment, the ﬂuctuation of performance is shown in Figure 7a.

Revised: When the reconstruction capability of AE is low, the model cannot complete the basic i.i.d.

Revised: task, and when it is high, the model may again prefer shortcuts.

Revised: Only the results in the middle section just meet our expectations.

Revised: Fortunately, we can avoid the ﬁrst case by requiring reasonable training accuracy.

Revised: And for the second case, we propose two simple strategies, on the one hand, we can consider the setting that just avoided the ﬁrst case; on the other hand, adjusting the weights of reconstruction loss can also guide models with high reconstruction capabilities to make reasonable trade-offs.

Revised: Let us change a perspective and discuss the behavior of AE.

Revised: It can be seen that with the expansion of the code layer, the accuracy on the i.i.d.

Revised: testing set continues to improve, and the accuracy on the o.o.d.

Revised: testing set increases ﬁrst and then decreases.

Revised: When the reconstruction capability is insufﬁcient,

Revised: AE tends to use attributes other than shortcuts, which is understandable because the roadblock loss is preventing AE from using shortcuts.

Revised: As the reconstruction capability improves, AE gradually learns shortcuts and weighs the shortcuts and other attributes under the guidance of losses.

Revised: When the ability to rebuild is greatly compromised, the weight of reconstruction loss also has impact on the training process.

Revised: We use hold the structure of BM and EM as ResNet-18, the weights of

Revised: [Figure]

Revised: other losses λ rb and λ ac are both set to 1.

Revised: The structure of AE is MLP, (512, 256, 512).

Revised: When other hyperparameters remain unchanged, the experimental results of different λ rec are shown in Figure 7b.

Revised: The settings of λ rec can be roughly divided into three types according to i.i.d.

Revised: set accuracy and o.o.d.

Revised: set accuracy.

Revised: (1) too high, Acc.

Revised: i.i.d. is close to 1, and Acc. o.o.d.

Revised: is close to 0, which means that AE tends to completely reconstruct original images, and EM can still learn shortcuts since roadblocks are invalid; (2) too low, Acc.

Revised: i.i.d. is low, and Acc. o.o.d.

Revised: is close to a random result, although roadblocks can work at this time, it is difﬁcult for EM to learn with reconstructed images; (3) appropriately,

Revised: Acc.

Revised: i.i.d. is good enough, and Acc. o.o.d.

Revised: is close to a random result, which indicates that EM has learned attributes other than shortcuts, and no longer relies too much on shortcuts while maintaining accuracy.

Revised: It can be seen that when λ rec is not too high, Acc. o.o.d.

Revised: is stable, so even if we may not be able to obtain o.o.d.

Revised: testing sets in real scenarios, it is not difﬁcult to ﬁnd suitable λ rec under the guidance of Acc.

Revised: i.i.d. .

Revised: A.4 Can adversarial training handle shortcuts?

Revised: We discuss the relationship of our method to adversarial training in the main text.

Revised: Table 4 is the performance of simple adversarial training with fast gradient sign untargeted adversarial attack [6] under the l2 constraint on CMNIST.

Revised: The maximum norm of the adversarial perturbation (cid:15) is 0.3, we use Adam as the optimizer, learning rate is 1e-2.

Revised: It can be seen that adversarial training does not have a signiﬁcant effect on biased datasets.

Revised: [Table]

Revised: We further conducted experiments under different adversarial training parameter settings, and the results are shown in Table 5.

Revised: It can be seen that the accuracy of the adversarial trained models on the adversarial samples and the clean samples are basically the same.

Revised: This is a demonstration that adversarial training has helped the model achieve a certain degree of adversarial robustness.

Revised: As epsilon increases, the accuracy of l2 improves, while the accuracy of linf increases ﬁrst, but it is difﬁcult to converge when epsilon is too large.

Revised: Some visualizations of adversarial examples shown in Figure 8 can help us understand why adversarial training has limited effect on the shortcut problem.

Revised: It can be seen that the difference between the adversarial sample and the real sample is almost invisible to the naked eye, and it is difﬁcult to observe clear digital information in the adversarial noise magniﬁed by 100 times.

Revised: We also conduct experiments on PGD [21], but it seems difﬁcult to obtain effective adversarial examples using MLPs.

Revised: So we use ResNet-18 as the backbone.

Revised: epsilon=0.03, k=4, alpha=0.03/4.

Revised: The accuracy is 92.91 on CMNIST (5pct) and 88.54 on adversarial examples.

Revised: (vanilla can achieve 93.34).

Revised: [Table]

Revised: [Figure]

Revised: Figure 9 shows some samples of PGD.

Revised: It can be seen that the noise of PGD seems to be more uniform compared to that of FGSM.

Revised: Neither can distinguish colors or numbers with the naked eye.

Revised: Based on these experimental results, we believe that simple adversarial training cannot solve the shortcut problem that this paper attempts to solve.

Revised: [Figure]

Revised: A.5 How to choose hyperparameters?

Revised: We tested different combinations of lambdas and selected the one that performed better, the result is shown in Figure 10.

Revised: For other hyperparameters, we did not tune too much, but adopted empirical settings.

Revised: A.6 More visualization results

Revised: We show more visualization experiments to demonstrate the generality of our conclusions.

Revised: Figure ??

Revised: is the result of CelebA and Figure 12 is the result of BAR.

Revised: For both of them, from top to bottom, is the original image, the reconstructed image, the difference, the saliency map of the original model, and the saliency map of EM.

Revised: 1e1e 1e 1e 1e3 rec 1e1e- 1e1e 1e r b 59.

Revised: We also show some selected results to try to analyze why our method can improve the generalization ability of the model.

Revised: Figure 13 is selected based on the predictions of BM and EM.

Revised: Green boxes indicate that the samples are classiﬁed correctly, and red boxes indicate the wrong predictions.

Revised: For action classiﬁcation, what should really be concerned with is the action of the person itself rather than other things in the scene.

Revised: Our method is able to better focus on the person, which results in better performance on the testing set where the environment changes.

Revised: For example, for the vanilla model, mountain climbing is misclassiﬁed as ﬁshing based on climbing ropes, and diving is misclassiﬁed as throwing based on spectator background, while EM can give the correct predictions.

Revised: In the proposed approach, AE and EM are alternately trained.

Revised: As the training continues, AE may modify the samples in different ways, as shown in Figure 14 image below.

Revised: The upper left corner is the original image, followed by the reconstruction results of AE at 10, 20, and 30 epochs.

Revised: A.7 More datasets results

Revised: We selected 10 categories of ImageNet to test the proposed approach in general scenarios.

Revised: Thecategories are n01608432, n01641577, n02106166, n02127052, n02389026, n02422106, n02690373,

Revised: Climbing

Revised: Diving

Revised: Fishing

Revised: Climbing

Revised: Diving

Revised: Fishing Climbing

Revised: Diving

Revised: Fishing

Revised: Climbing

Revised: Diving

Revised: Fishing

Revised: Climbing

Revised: Diving

Revised: Fishing

Revised: Fishing

Revised: Throwing

Revised: Diving Climbing

Revised: Diving

Revised: Fishing

Revised: Fishing

Revised: Throwing

Revised: Diving n03095699, n03100240, n03417042.

Revised: Figure 15 presents the corresponding reconstruction results and saliency maps.

Revised: [Table]

Revised: In the ImageNet-9 dataset, we chose to modify the background to generate a challenging testing set (mixed-rand, mixed-next), and the results are shown in Table 7 and Figure 16.

Revised: [Table]