Original: Roadblocks for Temporarily Disabling Shortcuts and Learning New Knowledge
Revised: Roadblocks for Temporarily Disabling Shortcuts and Learning New Knowledge

Original: Abstract
Revised: Abstract

Original: Deep learning models have been found with a tendency of relying on shortcuts, i.e., decision rules that perform well on standard benchmarks but fail when transferred to more challenging testing conditions.
Revised: Deep learning models have been found with a tendency of relying on shortcuts, i.e., decision rules that perform well on standard benchmarks but fail when transferred to more challenging testing conditions.

Original: Such reliance may hinder deep learning models from learning other task-related features and seriously affect their performance and robustness.
Revised: Such reliance may hinder deep learning models from learning other task-related features and seriously affect their performance and robustness.

Original: Although recent studies have shown some characteristics of shortcuts, there are few investigations on how to help the deep learning models to solve shortcut problems.
Revised: Although recent studies have shown some characteristics of shortcuts, there are few investigations on how to help the deep learning models to solve shortcut problems.

Original: This paper proposes a framework to address this issue by setting up roadblocks on shortcuts.
Revised: This paper proposes a framework to address this issue by setting up roadblocks on shortcuts.

Original: Speciﬁcally, roadblocks are placed when the model is urged to learn to complete a gently modiﬁed task to ensure that the learned knowledge, including shortcuts, is insufﬁcient the complete the task.
Revised: Speciﬁcally, roadblocks are placed when the model is urged to learn to complete a gently modiﬁed task to ensure that the learned knowledge, including shortcuts, is insufﬁcient the complete the task.

Original: Therefore, the model trained on the modiﬁed task will no longer over-rely on shortcuts.
Revised: Therefore, the model trained on the modiﬁed task will no longer over-rely on shortcuts.

Original: Extensive experiments demonstrate that the proposed framework signiﬁcantly improves the training of networks on both synthetic and real-world datasets in terms of both classiﬁcation accuracy and feature diversity.
Revised: Extensive experiments demonstrate that the proposed framework signiﬁcantly improves the training of networks on both synthetic and real-world datasets in terms of both classiﬁcation accuracy and feature diversity.

Original: Moreover, the visualization results show that the mechanism behind the proposed our method is consistent with our expectations.
Revised: Moreover, the visualization results show that the mechanism behind the proposed our method is consistent with our expectations.

Original: In summary, our approach can effectively disable the shortcuts and thus learn more robust features.
Revised: In summary, our approach can effectively disable the shortcuts and thus learn more robust features.

Original:  Introduction
Revised:  Introduction

Original: Deep learning has triggered the current rise of artiﬁcial intelligence and is the workhorse of today’s machine intelligence [13].
Revised: Deep learning has triggered the current rise of artiﬁcial intelligence and is the workhorse of today’s machine intelligence [13].

Original: One of the great advantages of deep learning is its ability to automatically extract the required features according to the task.
Revised: One of the great advantages of deep learning is its ability to automatically extract the required features according to the task.

Original: Moreover, the massive amount of data that drive the model is the key to its success.
Revised: Moreover, the massive amount of data that drive the model is the key to its success.

Original: Regrettably, deep learning methods can hardly take full advantage of the training data.
Revised: Regrettably, deep learning methods can hardly take full advantage of the training data.

Original: An important reason is its over-reliance on simple features.
Revised: An important reason is its over-reliance on simple features.

Original: Recently, deep learning approaches have been found to have a tendency to learn simple features only [4].
Revised: Recently, deep learning approaches have been found to have a tendency to learn simple features only [4].

Original: The learned simple features may be sufﬁcient for the accessible data (training data), and the model will have difﬁculty in learning other complex but effective features.
Revised: The learned simple features may be sufﬁcient for the accessible data (training data), and the model will have difﬁculty in learning other complex but effective features.

Original: Insufﬁcient learning of effective features may lower the accuracy and generalization performance.
Revised: Insufﬁcient learning of effective features may lower the accuracy and generalization performance.

Original: More severely, when the sample distribution changes and the learned simple features are no longer valid, deep learning models may collapse since they do not learn other features.
Revised: More severely, when the sample distribution changes and the learned simple features are no longer valid, deep learning models may collapse since they do not learn other features.

Original: Therefore, suppressing the reliance on simple features is indispensable for designing models.
Revised: Therefore, suppressing the reliance on simple features is indispensable for designing models.

Original: Shortcuts in deep learning are described as the decision rules that perform well on standard benchmarks but fail to be transferred to more challenging testing scenarios according to [4].
Revised: Shortcuts in deep learning are described as the decision rules that perform well on standard benchmarks but fail to be transferred to more challenging testing scenarios according to [4].

Original: We make a more speciﬁc explanation for easier evaluation.
Revised: We make a more speciﬁc explanation for easier evaluation.

Original: For a task, there are a number of features that should be considered.
Revised: For a task, there are a number of features that should be considered.

Original: Since the training set is limited, perhaps only a subset of features is sufﬁcient to make a decision.
Revised: Since the training set is limited, perhaps only a subset of features is sufﬁcient to make a decision.

Original: Decision rules that rely on an easy-to-learn subset of features are called shortcuts.
Revised: Decision rules that rely on an easy-to-learn subset of features are called shortcuts.

Original: The
Revised: The

Original: [Figure]
Revised: [Figure]

Original: Accessible:
Revised: Accessible:

Original: Inaccessible:
Revised: Inaccessible:

Original: [Figure]
Revised: [Figure]

Original: Inaccessible:
Revised: Inaccessible:

Original: [Figure]
Revised: [Figure]

Original: concept of shortcuts has two key points: (1) only use a subset of effective features but perform well on the training set; (2) easy to learn and affect the learning of other complex but effective features.
Revised: concept of shortcuts has two key points: (1) only use a subset of effective features but perform well on the training set; (2) easy to learn and affect the learning of other complex but effective features.

Original: Figure 1 provides an intuitive illustration of what shortcuts are and why shortcuts affect the generalization of deep learning models.
Revised: Figure 1 provides an intuitive illustration of what shortcuts are and why shortcuts affect the generalization of deep learning models.

Original: Each sample in Figure 1 has two features (color, shape).
Revised: Each sample in Figure 1 has two features (color, shape).

Original: The training set only has red circles and green triangles, while the testing set contains the other two combinations.
Revised: The training set only has red circles and green triangles, while the testing set contains the other two combinations.

Original: For the training set, decision rules are valid whether they are based on color or shape, but both of them may fail on the testing set because the training set does not articulate the intention.
Revised: For the training set, decision rules are valid whether they are based on color or shape, but both of them may fail on the testing set because the training set does not articulate the intention.

Original: A task may require the selection of a particular decision rule or a sum-up.
Revised: A task may require the selection of a particular decision rule or a sum-up.

Original: But either way, the diversity of decision rules is a foundation.
Revised: But either way, the diversity of decision rules is a foundation.

Original: Some studies expand the difference between decision rules at the feature level [10].
Revised: Some studies expand the difference between decision rules at the feature level [10].

Original: However, differentiated decision rules may still be functionally similar (see Figure 1a).
Revised: However, differentiated decision rules may still be functionally similar (see Figure 1a).

Original: To obtain the functionally diverse decision rules shown in Figure 1b, we propose to temporarily disable the shortcuts.
Revised: To obtain the functionally diverse decision rules shown in Figure 1b, we propose to temporarily disable the shortcuts.

Original: It is important to note that shortcuts are disabled temporarily, not forever.
Revised: It is important to note that shortcuts are disabled temporarily, not forever.

Original: We believe that simplicity does not mean bad, but more decision rules might help.
Revised: We believe that simplicity does not mean bad, but more decision rules might help.

Original: Our approach aims to simply provide more options.
Revised: Our approach aims to simply provide more options.

Original: The integration of decision rules should depend on the task scenarios, and we also provide some suggestions.
Revised: The integration of decision rules should depend on the task scenarios, and we also provide some suggestions.

Original: To address the issue that deep learning methods do not learn anything other than shortcuts, we propose roadblocks to temporarily disable the shortcuts and force the model to learn new knowledge.
Revised: To address the issue that deep learning methods do not learn anything other than shortcuts, we propose roadblocks to temporarily disable the shortcuts and force the model to learn new knowledge.

Original: The ability of roadblocks that disables the shortcuts is achieved by reconstructing datasets under the guidance of trained models.
Revised: The ability of roadblocks that disables the shortcuts is achieved by reconstructing datasets under the guidance of trained models.

Original: A reconstructed sample should retain most of the attributes of the original sample while disabling the learned attributes.
Revised: A reconstructed sample should retain most of the attributes of the original sample while disabling the learned attributes.

Original: Thus, the model trained with reconstructed samples will naturally reduce the reliance on shortcuts and learn new knowledge.
Revised: Thus, the model trained with reconstructed samples will naturally reduce the reliance on shortcuts and learn new knowledge.

Original: Enlarging the distance directly [10] at the feature level often only learns the decision rules shown in
Revised: Enlarging the distance directly [10] at the feature level often only learns the decision rules shown in

Original: Figure 1a, but it is difﬁcult to learn the functional differences.
Revised: Figure 1a, but it is difﬁcult to learn the functional differences.

Original: We believe the reason is the redundancy and weak interpretability of deep learning models.
Revised: We believe the reason is the redundancy and weak interpretability of deep learning models.

Original: There is currently no metric that can measure the functional differences of the models well.
Revised: There is currently no metric that can measure the functional differences of the models well.

Original: Most previous researches on shortcuts focus on what features are more likely to be shortcuts, and the corresponding solutions mostly rely on the prior knowledge of shortcuts [5, 8, 18].
Revised: Most previous researches on shortcuts focus on what features are more likely to be shortcuts, and the corresponding solutions mostly rely on the prior knowledge of shortcuts [5, 8, 18].

Original: Priori-based methods require accurate prior knowledge and targeted design, which are difﬁcult to accomplish in most scenarios.
Revised: Priori-based methods require accurate prior knowledge and targeted design, which are difﬁcult to accomplish in most scenarios.

Original: Unlike them, our roadblocks do not use any prior knowledge about shortcuts and can deal with different forms of shortcuts.
Revised: Unlike them, our roadblocks do not use any prior knowledge about shortcuts and can deal with different forms of shortcuts.

Original: In summary, the contributions of this work are mainly three-fold:
Revised: In summary, the contributions of this work are mainly three-fold:

Original: • We propose a new framework to help models learn more diverse knowledge.
Revised: • We propose a new framework to help models learn more diverse knowledge.

Original: Compared with those methods based on distance at the feature level, our approach can learn brand-new knowledge and obtain functionally diverse decision rules.
Revised: Compared with those methods based on distance at the feature level, our approach can learn brand-new knowledge and obtain functionally diverse decision rules.

Original: • We design a new strategy for disabling shortcuts, which requires no priors and is suitable for a wider range of application scenarios.
Revised: • We design a new strategy for disabling shortcuts, which requires no priors and is suitable for a wider range of application scenarios.

Original: • Our method can effectively improve the generalization ability of the model and can reach state-of-the-arts in the debiasing task.
Revised: • Our method can effectively improve the generalization ability of the model and can reach state-of-the-arts in the debiasing task.

Original: [Figure]
Revised: [Figure]

Original:  Method
Revised:  Method

Original: This section will introduce our strategy to address the shortcuts problem, which is temporarily disabling the shortcuts and learning more diverse knowledge.
Revised: This section will introduce our strategy to address the shortcuts problem, which is temporarily disabling the shortcuts and learning more diverse knowledge.

Original: First, we specify some deﬁnitions and the evaluation system.
Revised: First, we specify some deﬁnitions and the evaluation system.

Original: Then, we introduce our roadblocks framework.
Revised: Then, we introduce our roadblocks framework.

Original: Finally, the working process of the framework will be detailed.
Revised: Finally, the working process of the framework will be detailed.

Original: 2.1 Shortcut Suppression Task
Revised: 2.1 Shortcut Suppression Task

Original: Shortcuts are described as decision rules that perform well on independently-identical-distribution (i.i.d.) testing sets but fail to generalize to out-of-distribution (o.o.d.) testing sets [4].
Revised: Shortcuts are described as decision rules that perform well on independently-identical-distribution (i.i.d.) testing sets but fail to generalize to out-of-distribution (o.o.d.) testing sets [4].

Original: We refer to this idea and describe the shortcut more precisely, shortcuts are decision rules that rely on an easy-to-learn subset of features.
Revised: We refer to this idea and describe the shortcut more precisely, shortcuts are decision rules that rely on an easy-to-learn subset of features.

Original: For classiﬁcation tasks, suppose a distribution D whose feature space can be represented by F .
Revised: For classiﬁcation tasks, suppose a distribution D whose feature space can be represented by F .

Original: Supposing that a training set D train is sampled from D , and there exist a decision rule that based on F train that can perform well on D train .
Revised: Supposing that a training set D train is sampled from D , and there exist a decision rule that based on F train that can perform well on D train .

Original: F train is a subspace of F .
Revised: F train is a subspace of F .

Original: Such a decision rule has the potential risk of failing in other datasets sampled from D because it ignores other features in F .
Revised: Such a decision rule has the potential risk of failing in other datasets sampled from D because it ignores other features in F .

Original: For ease of measurement, we refer to the setting of the debiasing task.
Revised: For ease of measurement, we refer to the setting of the debiasing task.

Original: The feature space contains two main features, F =
Revised: The feature space contains two main features, F =

Original: [ F T , F B ] .
Revised: [ F T , F B ] .

Original: In the training data, the target feature F T corresponds to the ground truth perfectly; while biased feature F B corresponds to the ground truth with a large probability.
Revised: In the training data, the target feature F T corresponds to the ground truth perfectly; while biased feature F B corresponds to the ground truth with a large probability.

Original: Nonetheless, there is still a possibility that F train is closer to F B because F B is simpler, which means that the model may not have learned F T as we expected.
Revised: Nonetheless, there is still a possibility that F train is closer to F B because F B is simpler, which means that the model may not have learned F T as we expected.

Original: To verify the learning of F T in the testing data, F T still corresponds to the ground truth perfectly, and F B is no longer helpful.
Revised: To verify the learning of F T in the testing data, F T still corresponds to the ground truth perfectly, and F B is no longer helpful.

Original: 2.2 Framework of Roadblocks
Revised: 2.2 Framework of Roadblocks

Original: Assuming that existing models use shortcuts, a strategy to ﬁnd more reasonable decision rules is to use the idea of roadblocks that forces the new model not to use the same decision rule as any previous model.
Revised: Assuming that existing models use shortcuts, a strategy to ﬁnd more reasonable decision rules is to use the idea of roadblocks that forces the new model not to use the same decision rule as any previous model.

Original: Figure 2 illustrates the framework of our approach.
Revised: Figure 2 illustrates the framework of our approach.

Original: The overall model is composed of three parts.
Revised: The overall model is composed of three parts.

Original: Blocked models are trained models that perform well on the i.i.d.
Revised: Blocked models are trained models that perform well on the i.i.d.

Original: test dataset.
Revised: test dataset.

Original: The autoencoder is used to modify the original input, thereby limiting the performance of blocked models.
Revised: The autoencoder is used to modify the original input, thereby limiting the performance of blocked models.

Original: Once the blocked models perform poorly on the modiﬁed input, the explorer models trained with the modiﬁed input will ﬁnd new ways to achieve competitive performance.
Revised: Once the blocked models perform poorly on the modiﬁed input, the explorer models trained with the modiﬁed input will ﬁnd new ways to achieve competitive performance.

Original: None of the three sub-models need a speciﬁc structure, as long as they have sufﬁcient representation capabilities.
Revised: None of the three sub-models need a speciﬁc structure, as long as they have sufﬁcient representation capabilities.

Original: Let AE, BM, EM denote autoencoder, blocked model, explorer model, respectively.
Revised: Let AE, BM, EM denote autoencoder, blocked model, explorer model, respectively.

Original: BM is a pretrained model with good performance, which is ﬁxed during the training process.
Revised: BM is a pretrained model with good performance, which is ﬁxed during the training process.

Original: AE receives the raw input I raw and modiﬁes it into blocked input I rec .
Revised: AE receives the raw input I raw and modiﬁes it into blocked input I rec .

Original: The mean square error (MSE) is used to measure the similarity between them,
Revised: The mean square error (MSE) is used to measure the similarity between them,

Original: [Equation]
Revised: [Equation]

Original: The reconstructed input needs to meet two requirements, (1) it is difﬁcult to distinguish with the existing decision rules, and (2) it can be correctly distinguished.
Revised: The reconstructed input needs to meet two requirements, (1) it is difﬁcult to distinguish with the existing decision rules, and (2) it can be correctly distinguished.

Original: I rec will ﬁrst be used as the input of BM.
Revised: I rec will ﬁrst be used as the input of BM.

Original: The MSE of the prediction BM ( I rec ) and the pseudo label label ep will be used as the roadblock loss,
Revised: The MSE of the prediction BM ( I rec ) and the pseudo label label ep will be used as the roadblock loss,

Original: [Equation]
Revised: [Equation]

Original: The pseudo label label ep is set to consider all possible categories with equal probability, which is label ep =
Revised: The pseudo label label ep is set to consider all possible categories with equal probability, which is label ep =

Original: [ 1 n , 1 n , ..., 1 n ]) , where n is the number of categories.
Revised: [ 1 n , 1 n , ..., 1 n ]) , where n is the number of categories.

Original: In this way, the purpose of AE is to generate I rec to confuse BM.
Revised: In this way, the purpose of AE is to generate I rec to confuse BM.

Original: I rec will also be used as the input of EM.
Revised: I rec will also be used as the input of EM.

Original: In order for the EM to achieve better classiﬁcation performance, the accuracy loss between the prediction EM ( I rec ) and the ground-truth label is calculated for training,
Revised: In order for the EM to achieve better classiﬁcation performance, the accuracy loss between the prediction EM ( I rec ) and the ground-truth label is calculated for training,

Original: [Equation]
Revised: [Equation]

Original: 2.3 Working Process
Revised: 2.3 Working Process

Original: Core training process BM is a pre-trained model, and it does not participate in the training process of the proposed framework.
Revised: Core training process BM is a pre-trained model, and it does not participate in the training process of the proposed framework.

Original: Both AE and EM need to be trained.
Revised: Both AE and EM need to be trained.

Original: We alternately train them for a more stable training process.
Revised: We alternately train them for a more stable training process.

Original: A typical training cycle consists of two parts, (1) AE is trained with roadblock loss and reconstruction loss; (2) EM is trained under the guidance of accuracy loss,
Revised: A typical training cycle consists of two parts, (1) AE is trained with roadblock loss and reconstruction loss; (2) EM is trained under the guidance of accuracy loss,

Original: [Equation]
Revised: [Equation]

Original: Here, θ AE and θ EM denote the parameters of AE and EM respectively, and λ =
Revised: Here, θ AE and θ EM denote the parameters of AE and EM respectively, and λ =

Original: [ λ rb , λ rec , λ ac ] are the corresponding loss weights.
Revised: [ λ rb , λ rec , λ ac ] are the corresponding loss weights.

Original: Multi-model BM can be any model that performs well on training set.
Revised: Multi-model BM can be any model that performs well on training set.

Original: In addition to vanilla models, a EM generated by roadblocks method can also be used as BM to train a new EM.
Revised: In addition to vanilla models, a EM generated by roadblocks method can also be used as BM to train a new EM.

Original: BM can also be a set of models BM =
Revised: BM can also be a set of models BM =

Original: [ BM 1 , BM 2 , ..., BM n ] .
Revised: [ BM 1 , BM 2 , ..., BM n ] .

Original: Correspondingly, the roadblock loss needs to be adjusted to
Revised: Correspondingly, the roadblock loss needs to be adjusted to

Original: [Equation]
Revised: [Equation]

Original: We suggest adding the well-trained EM to the set of BM and then training the new EM.
Revised: We suggest adding the well-trained EM to the set of BM and then training the new EM.

Original: When the new EM fails to perform well on the training set, it means that it is difﬁcult to learn more effective features, at which point the loop terminates.
Revised: When the new EM fails to perform well on the training set, it means that it is difﬁcult to learn more effective features, at which point the loop terminates.

Original: The experiments suggest that in most cases, two models are sufﬁcient.
Revised: The experiments suggest that in most cases, two models are sufﬁcient.

Original: Therefore, to reduce the computational cost, we usually use two models.
Revised: Therefore, to reduce the computational cost, we usually use two models.

Original: Sum-up The proposed framework contains multiple sub-models that learn different effective features respectively.
Revised: Sum-up The proposed framework contains multiple sub-models that learn different effective features respectively.

Original: We do not recommend discarding any features directly, as the ease of learning and the effectiveness of features are not necessarily related.
Revised: We do not recommend discarding any features directly, as the ease of learning and the effectiveness of features are not necessarily related.

Original: We recommend designing according to speciﬁc scenarios.
Revised: We recommend designing according to speciﬁc scenarios.

Original: Without priors on the test scene, we recommend using the feature fusion strategy because it is more likely to have good generalization.
Revised: Without priors on the test scene, we recommend using the feature fusion strategy because it is more likely to have good generalization.

Original: Another idea is to vote, and o.o.d.
Revised: Another idea is to vote, and o.o.d.

Original: samples can be detected if there is a signiﬁcant difference in the votes of the submodels.
Revised: samples can be detected if there is a signiﬁcant difference in the votes of the submodels.

Original: For some tasks with priors, the appropriate model can be directly selected.
Revised: For some tasks with priors, the appropriate model can be directly selected.

Original: For example, for debiasing tasks, a reasonable idea is to choose the model that performs better on the training set because the biased features are less matched with the ground truth.
Revised: For example, for debiasing tasks, a reasonable idea is to choose the model that performs better on the training set because the biased features are less matched with the ground truth.

Original:  Experiments
Revised:  Experiments

Original: To verify whether functionally different decision rules are learned, we choose datasets with multiple attribute labels for experiments.
Revised: To verify whether functionally different decision rules are learned, we choose datasets with multiple attribute labels for experiments.

Original: The experimental setup refers to the design of the debiasing task [15].
Revised: The experimental setup refers to the design of the debiasing task [15].

Original: Each individual attribute in the training set corresponds well to the ground truth, while only one attribute matches the ground truth one-to-one in the testing set.
Revised: Each individual attribute in the training set corresponds well to the ground truth, while only one attribute matches the ground truth one-to-one in the testing set.

Original: Taking Figure 1 as an example,
Revised: Taking Figure 1 as an example,

Original: [Table] [Table]
Revised: [Table] [Table]

Original: the training set is mainly red circles and green triangles, and the four combinations have the same proportion in the testing set.
Revised: the training set is mainly red circles and green triangles, and the four combinations have the same proportion in the testing set.

Original: We use the synthetic dataset CMNIST [15], which adds a second attribute by coloring MNIST [14].
Revised: We use the synthetic dataset CMNIST [15], which adds a second attribute by coloring MNIST [14].

Original: We also use real-world datasets CelebA [20], and BAR [23] that were validated to have shortcuts to test the practicality of our method.
Revised: We also use real-world datasets CelebA [20], and BAR [23] that were validated to have shortcuts to test the practicality of our method.

Original: Details of datasets and implementation are described in the appendix.
Revised: Details of datasets and implementation are described in the appendix.

Original: Extensive experiments have been conducted to demonstrate the following effects of roadblocks: 1) greatly enhance the ability to explore new knowledge, 2) signiﬁcantly improve the generalization of deep learning models.
Revised: Extensive experiments have been conducted to demonstrate the following effects of roadblocks: 1) greatly enhance the ability to explore new knowledge, 2) signiﬁcantly improve the generalization of deep learning models.

Original: 3.1 Exploration of Knowledge
Revised: 3.1 Exploration of Knowledge

Original: To verify whether roadblocks can reduce models’ reliance on shortcuts, we compared the performance of the conventionally trained CNNs (Vanilla) and some representative debiasing methods and our roadblocks method.
Revised: To verify whether roadblocks can reduce models’ reliance on shortcuts, we compared the performance of the conventionally trained CNNs (Vanilla) and some representative debiasing methods and our roadblocks method.

Original: We ﬁrst test on a controlled datasets CMNIST to facilitate analysis.
Revised: We ﬁrst test on a controlled datasets CMNIST to facilitate analysis.

Original: The experimental results are shown in Table 1.
Revised: The experimental results are shown in Table 1.

Original: For CMNIST, each sample has two key features, color, and digit.
Revised: For CMNIST, each sample has two key features, color, and digit.

Original: Although the colors are slightly less matched with ground truth, deep learning models tend to learn colors.
Revised: Although the colors are slightly less matched with ground truth, deep learning models tend to learn colors.

Original: Our method can help deep learning models learn more decision rules that utilize digit features.
Revised: Our method can help deep learning models learn more decision rules that utilize digit features.

Original: Models also often rely on shortcuts to complete their tasks in real-world scenarios.
Revised: Models also often rely on shortcuts to complete their tasks in real-world scenarios.

Original: We conducted experiments on the face attribute dataset CelebA [20] and the action recognition dataset BAR [23].
Revised: We conducted experiments on the face attribute dataset CelebA [20] and the action recognition dataset BAR [23].

Original: For real-world datasets, we keep its native ratio to maintain the authenticity during the training process.
Revised: For real-world datasets, we keep its native ratio to maintain the authenticity during the training process.

Original: And then we test whether the model learned more knowledge through testing sets of different distributions.
Revised: And then we test whether the model learned more knowledge through testing sets of different distributions.

Original: For CelebA, following the experimental setup of LFF [23], we trained models to classify ‘HeavyMakeup’.
Revised: For CelebA, following the experimental setup of LFF [23], we trained models to classify ‘HeavyMakeup’.

Original: Then we select the same number of samples from all possible combinations of [Gender, HeavyMakeup] to form the unbiased testing set and minority combinations to form the bias-conﬂicting testing set.
Revised: Then we select the same number of samples from all possible combinations of [Gender, HeavyMakeup] to form the unbiased testing set and minority combinations to form the bias-conﬂicting testing set.

Original: BAR contains six actions, and the backgrounds of the training and testing sets are signiﬁcantly different.
Revised: BAR contains six actions, and the backgrounds of the training and testing sets are signiﬁcantly different.

Original: The testing set has only bias-conﬂict samples.
Revised: The testing set has only bias-conﬂict samples.

Original: The results are shown in Table 2.
Revised: The results are shown in Table 2.

Original: It can be seen that our method has obvious performance improvement compared to the state-of-the-art methods.
Revised: It can be seen that our method has obvious performance improvement compared to the state-of-the-art methods.

Original: In particular, for the bias-conﬂict dataset which is more difﬁcult, our method achieves more than 10% improvement.
Revised: In particular, for the bias-conﬂict dataset which is more difﬁcult, our method achieves more than 10% improvement.

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: 3.2 Disabling Shortcuts
Revised: 3.2 Disabling Shortcuts

Original: The above experiments veriﬁed the effectiveness of our roadblocks framework in learning functionally diverse decision rules.
Revised: The above experiments veriﬁed the effectiveness of our roadblocks framework in learning functionally diverse decision rules.

Original: Next, it is necessary to conﬁrm whether the effective mechanism of the proposed method is consistent with the original intention of the design.
Revised: Next, it is necessary to conﬁrm whether the effective mechanism of the proposed method is consistent with the original intention of the design.

Original: Speciﬁcally, it is necessary to verify that some information in the reconstructed image is indeed corrupted so that the existing decision rules cannot make correct predictions.
Revised: Speciﬁcally, it is necessary to verify that some information in the reconstructed image is indeed corrupted so that the existing decision rules cannot make correct predictions.

Original: Thus, we visualized reconstructed images, and the result is shown in Figure 3.
Revised: Thus, we visualized reconstructed images, and the result is shown in Figure 3.

Original: It can be observed that for CMNIST, our method can implement the disabling of the shortcut (color) by repainting digits.
Revised: It can be observed that for CMNIST, our method can implement the disabling of the shortcut (color) by repainting digits.

Original: For different digits, the strategy of repainting is consistent, which shows that only the color attribute is disabled in the painting stage, while the number attribute is retained.
Revised: For different digits, the strategy of repainting is consistent, which shows that only the color attribute is disabled in the painting stage, while the number attribute is retained.

Original: Therefore, it is still feasible to use these samples to learn numbers, which is also well illustrated in Table 1.
Revised: Therefore, it is still feasible to use these samples to learn numbers, which is also well illustrated in Table 1.

Original: For real-world datasets, similar modiﬁcation methods are challenging to achieve, and our approach mainly implements the destruction by adding noise.
Revised: For real-world datasets, similar modiﬁcation methods are challenging to achieve, and our approach mainly implements the destruction by adding noise.

Original: For CelebA, AE tries to blur the gender attribute by some blurring (mainly focused on the face contour).
Revised: For CelebA, AE tries to blur the gender attribute by some blurring (mainly focused on the face contour).

Original: For BAR, the blurring of AE mainly revolves around the environment.
Revised: For BAR, the blurring of AE mainly revolves around the environment.

Original: For the convenience of observation, we show the difference map of the original image and the reconstructed image, which is ﬁve times the absolute value of the difference of each pixel value between the two.
Revised: For the convenience of observation, we show the difference map of the original image and the reconstructed image, which is ﬁve times the absolute value of the difference of each pixel value between the two.

Original: We also used t-SNE [30] to visualize the features extracted by the vanilla model and our approach, and the result is shown in Figure 4.
Revised: We also used t-SNE [30] to visualize the features extracted by the vanilla model and our approach, and the result is shown in Figure 4.

Original: Each colored number represents a sample.
Revised: Each colored number represents a sample.

Original: It is evident that the vanilla model does an excellent job of partitioning the dataset according to color.
Revised: It is evident that the vanilla model does an excellent job of partitioning the dataset according to color.

Original: However, a closer look reveals that samples of the same color, regardless of the digit, are in the same cluster.
Revised: However, a closer look reveals that samples of the same color, regardless of the digit, are in the same cluster.

Original: In contrast, the model trained by our approach can distinguish digits and have certain invariance to color.
Revised: In contrast, the model trained by our approach can distinguish digits and have certain invariance to color.

Original: Distinguishing digits is more complex than color, so getting sharp boundaries is more complicated.
Revised: Distinguishing digits is more complex than color, so getting sharp boundaries is more complicated.

Original: To visually demonstrate the model’s ability to explore new knowledge, we use Grad-CAM [26] to visualize the focus of models on the BAR dataset, and saliency maps are shown in Figure 5.
Revised: To visually demonstrate the model’s ability to explore new knowledge, we use Grad-CAM [26] to visualize the focus of models on the BAR dataset, and saliency maps are shown in Figure 5.

Original: In this experiment, we use ResNet-18 as the backbone, and the last fully connected layer is modiﬁed to a linear layer (512, 10) to match the number of categories.
Revised: In this experiment, we use ResNet-18 as the backbone, and the last fully connected layer is modiﬁed to a linear layer (512, 10) to match the number of categories.

Original: And the last convolutional layer is used as the target layer of Grad-CAM.
Revised: And the last convolutional layer is used as the target layer of Grad-CAM.

Original: For CMNIST, it is difﬁcult to tell the difference from the saliency map because the colors and shapes correspond to the same regions.
Revised: For CMNIST, it is difﬁcult to tell the difference from the saliency map because the colors and shapes correspond to the same regions.

Original: This is also the reason why we do not disable shortcuts using saliency map based methods.
Revised: This is also the reason why we do not disable shortcuts using saliency map based methods.

Original: For CelebA and BAR, it can be seen that the attention regions of our approach are signiﬁcantly different from that of the vanilla model.
Revised: For CelebA and BAR, it can be seen that the attention regions of our approach are signiﬁcantly different from that of the vanilla model.

Original: For correctly classiﬁed samples, our approach can ﬁnd more effective information.
Revised: For correctly classiﬁed samples, our approach can ﬁnd more effective information.

Original: Grad-CAM also provides some explanation for samples that vanilla misclassiﬁes, but our method classiﬁes correctly.
Revised: Grad-CAM also provides some explanation for samples that vanilla misclassiﬁes, but our method classiﬁes correctly.

Original: For example, for samples of mountain climbing, the EM of our approach focuses on the climber, and the vanilla model focuses on the mountain.
Revised: For example, for samples of mountain climbing, the EM of our approach focuses on the climber, and the vanilla model focuses on the mountain.

Original: Both of these features are important, and our approach has better generalization ability because it utilizes more effective features.
Revised: Both of these features are important, and our approach has better generalization ability because it utilizes more effective features.

Original: The samples of CMNIST and CelebA were randomly selected, and we selected more representative samples for BAR.
Revised: The samples of CMNIST and CelebA were randomly selected, and we selected more representative samples for BAR.

Original: There
Revised: There

Original: [Figure]
Revised: [Figure]

Original: are more examples of samples in the Appendix to facilitate more analysis and discussion, while also ensuring generality.
Revised: are more examples of samples in the Appendix to facilitate more analysis and discussion, while also ensuring generality.

Original: Grad-CAM is an excellent interpretability method, but it can only show differences in models at a spatial scale.
Revised: Grad-CAM is an excellent interpretability method, but it can only show differences in models at a spatial scale.

Original: Therefore, for datasets such as CMNIST, where different features overlap a lot in spatial scales, saliency maps cannot well reﬂect the differences between models.
Revised: Therefore, for datasets such as CMNIST, where different features overlap a lot in spatial scales, saliency maps cannot well reﬂect the differences between models.

Original: The reconstruction results and saliency maps visually demonstrate that our approach does learn more effective features following our design intention.
Revised: The reconstruction results and saliency maps visually demonstrate that our approach does learn more effective features following our design intention.

Original: These methods still have some limitations.
Revised: These methods still have some limitations.

Original: To verify the ability of our method to improve model diversity as fully as possible, we further quantify the model diversity.
Revised: To verify the ability of our method to improve model diversity as fully as possible, we further quantify the model diversity.

Original: 3.3 Model Diversity
Revised: 3.3 Model Diversity

Original: Our core purpose is to learn more knowledge (i.e., effective features).
Revised: Our core purpose is to learn more knowledge (i.e., effective features).

Original: Therefore, it is necessary to choose appropriate metrics to measure the similarity between models.
Revised: Therefore, it is necessary to choose appropriate metrics to measure the similarity between models.

Original: We applied SVCCA [24] to measure the similarity of the models.
Revised: We applied SVCCA [24] to measure the similarity of the models.

Original: SVCCA is a tool for comparing two representations in a way that is invariant to afﬁne transform.
Revised: SVCCA is a tool for comparing two representations in a way that is invariant to afﬁne transform.

Original: It receives the feature maps of middle layers and calculates the similarity between each pair of feature maps.
Revised: It receives the feature maps of middle layers and calculates the similarity between each pair of feature maps.

Original: Figure 6 shows some SVCCA results between the models.
Revised: Figure 6 shows some SVCCA results between the models.

Original: Figure 6a and Figure 6b are calculated on CMNIST (5pct), Figure 6c and Figure 6d are calculated on BAR.
Revised: Figure 6a and Figure 6b are calculated on CMNIST (5pct), Figure 6c and Figure 6d are calculated on BAR.

Original: CelebA is too large to be calculated, so we did not perform corresponding experiments.
Revised: CelebA is too large to be calculated, so we did not perform corresponding experiments.

Original: Vanilla
Revised: Vanilla

Original: V a n ill a (a) CMNIST, V-V
Revised: V a n ill a (a) CMNIST, V-V

Original: Ours
Revised: Ours

Original: V a n ill a (b) CMNIST, V-O
Revised: V a n ill a (b) CMNIST, V-O

Original: Vanilla
Revised: Vanilla

Original: V a n ill a (c) BAR, V-V
Revised: V a n ill a (c) BAR, V-V

Original: Ours
Revised: Ours

Original: V a n ill a (d) BAR, V-O
Revised: V a n ill a (d) BAR, V-O

Original: For the CMNIST dataset, the diagonal of the SVCCA matrix of the vanilla model is bright, indicating a high similarity between the same layers.
Revised: For the CMNIST dataset, the diagonal of the SVCCA matrix of the vanilla model is bright, indicating a high similarity between the same layers.

Original: While the similarity between our model and the vanilla model is slight, there are only a few brighter regions between several lower layers.
Revised: While the similarity between our model and the vanilla model is slight, there are only a few brighter regions between several lower layers.

Original: The comparison of the two shows that the model trained by our method is signiﬁcantly different from the vanilla model at the feature level.
Revised: The comparison of the two shows that the model trained by our method is signiﬁcantly different from the vanilla model at the feature level.

Original: Therefore, our method helps to improve the diversity of models.
Revised: Therefore, our method helps to improve the diversity of models.

Original: It can also be observed that some adjacent layers also have high similarities.
Revised: It can also be observed that some adjacent layers also have high similarities.

Original: The BAR dataset is more complex than the CMNIST dataset.
Revised: The BAR dataset is more complex than the CMNIST dataset.

Original: Therefore, the similarity between adjacent layers in the vanilla model is lower.
Revised: Therefore, the similarity between adjacent layers in the vanilla model is lower.

Original: Some layers of our model and the corresponding layers vanilla model are similar, which we think may correspond to some complex semantic features.
Revised: Some layers of our model and the corresponding layers vanilla model are similar, which we think may correspond to some complex semantic features.

Original:  Related Work and Discussion 4.1 Related Work
Revised:  Related Work and Discussion 4.1 Related Work

Original: Shortcut learning .
Revised: Shortcut learning .

Original: Geirhos et al.
Revised: Geirhos et al.

Original: [4] provided a comprehensive summary of shortcuts and deﬁned them as decision rules that perform poorly on the o.o.d.
Revised: [4] provided a comprehensive summary of shortcuts and deﬁned them as decision rules that perform poorly on the o.o.d.

Original: testing set.
Revised: testing set.

Original: A considerable amount of research has been devoted to analyzing the formation of shortcuts, providing many valuable conclusions [27, 22].
Revised: A considerable amount of research has been devoted to analyzing the formation of shortcuts, providing many valuable conclusions [27, 22].

Original: Deep learning models is found to have the tendency to use textures rather than shapes to make predictions [5], which has inspired a new round of thinking among researchers on the mechanism of deep learning.
Revised: Deep learning models is found to have the tendency to use textures rather than shapes to make predictions [5], which has inspired a new round of thinking among researchers on the mechanism of deep learning.

Original: Katherine and Andrew [9] explored the conditions under which features are more likely to be shortcuts, which is very helpful to deepen our understanding of shortcuts.
Revised: Katherine and Andrew [9] explored the conditions under which features are more likely to be shortcuts, which is very helpful to deepen our understanding of shortcuts.

Original: In addition, various data augmentation methods are used to drive models to learn features beyond known shortcuts.
Revised: In addition, various data augmentation methods are used to drive models to learn features beyond known shortcuts.

Original: Some researches [5, 8, 18] assumed that the texture feature is the only shortcut, and the designed data augmentation techniques effectively reduce the model’s reliance on texture.
Revised: Some researches [5, 8, 18] assumed that the texture feature is the only shortcut, and the designed data augmentation techniques effectively reduce the model’s reliance on texture.

Original: There are also ways to exclude shortcuts by introducing a prior on the difﬁculty of the target task [3].
Revised: There are also ways to exclude shortcuts by introducing a prior on the difﬁculty of the target task [3].

Original: However, research on suppressing shortcut without relying on prior knowledge is still basically an unknown area to explore.
Revised: However, research on suppressing shortcut without relying on prior knowledge is still basically an unknown area to explore.

Original: Fairness and debiasing .
Revised: Fairness and debiasing .

Original: The debiasing task can be understood as solving the problem that the model has learned a certain bias attribute in the data set, but failed to learn the relatively difﬁcult target attribute.
Revised: The debiasing task can be understood as solving the problem that the model has learned a certain bias attribute in the data set, but failed to learn the relatively difﬁcult target attribute.

Original: Some researchers have explored the preferences of deep learning methods themselves and dealt with them accordingly, such as the tendency to use textures rather than shapes [5] and the tendency to learn low-frequency features in images [2].
Revised: Some researchers have explored the preferences of deep learning methods themselves and dealt with them accordingly, such as the tendency to use textures rather than shapes [5] and the tendency to learn low-frequency features in images [2].

Original: Besides, several existing approaches aimed at mitigating the bias also assume certain bias types [12, 31, 17, 29, 25].
Revised: Besides, several existing approaches aimed at mitigating the bias also assume certain bias types [12, 31, 17, 29, 25].

Original: Instead of deﬁning speciﬁc types of bias, some approaches [23, 15, 19, 16] rely on the straightforward assumption that simple features are biased features.
Revised: Instead of deﬁning speciﬁc types of bias, some approaches [23, 15, 19, 16] rely on the straightforward assumption that simple features are biased features.

Original: A typical example is Learn from Failure .
Revised: A typical example is Learn from Failure .

Original: The main idea is twofold; (a) intentionally train the ﬁrst network to be biased by repeatedly amplifying its “prejudice”, and (b) debias the training of the second network by focusing on samples that go against the prejudice of the biased network in (a).
Revised: The main idea is twofold; (a) intentionally train the ﬁrst network to be biased by repeatedly amplifying its “prejudice”, and (b) debias the training of the second network by focusing on samples that go against the prejudice of the biased network in (a).

Original: Unlike them, our approach does not use biased priors, but solves the debiasing problem by learning more knowledge.
Revised: Unlike them, our approach does not use biased priors, but solves the debiasing problem by learning more knowledge.

Original: Compared with other debiasing methods, it has a wider range of application scenarios.
Revised: Compared with other debiasing methods, it has a wider range of application scenarios.

Original: Adversarial training The proposed roadblocks share some similarities with adversarial training.
Revised: Adversarial training The proposed roadblocks share some similarities with adversarial training.

Original: Adversarial examples obtained by adding small designed perturbations can make the model go wrong [28].
Revised: Adversarial examples obtained by adding small designed perturbations can make the model go wrong [28].

Original: Adversarial training [11] can effectively improve the adversarial robustness of the model.
Revised: Adversarial training [11] can effectively improve the adversarial robustness of the model.

Original: Our approach and adversarial training clearly differ in purpose, which also introduces design differences.
Revised: Our approach and adversarial training clearly differ in purpose, which also introduces design differences.

Original: Adversarial examples only require the model to predict incorrectly, whereas our method requires the model to be completely confused.
Revised: Adversarial examples only require the model to predict incorrectly, whereas our method requires the model to be completely confused.

Original: Therefore, by design, the false labels we use are all categories, including the true category, with equal probability.
Revised: Therefore, by design, the false labels we use are all categories, including the true category, with equal probability.

Original: We also conduct experiments using adversarial training, and the results are shown in the Appendix.
Revised: We also conduct experiments using adversarial training, and the results are shown in the Appendix.

Original: 4.2 Discussion
Revised: 4.2 Discussion

Original: Functionally identical features may have different representations.
Revised: Functionally identical features may have different representations.

Original: Increasing the diversity of features, in other words, learning more knowledge, is a natural idea, but its implementation is not as simple.
Revised: Increasing the diversity of features, in other words, learning more knowledge, is a natural idea, but its implementation is not as simple.

Original: An intuitive way to learn different features is to expand the distance between the two models at the feature level.
Revised: An intuitive way to learn different features is to expand the distance between the two models at the feature level.

Original: We refer to the framework of negative correlation learning and expand the distance between models at the feature level as much as possible on the premise of ensuring consistent prediction results.
Revised: We refer to the framework of negative correlation learning and expand the distance between models at the feature level as much as possible on the premise of ensuring consistent prediction results.

Original: Experimental results (see the Appendix) show that it is difﬁcult to learn decision rules other than shortcuts through such a learning paradigm.
Revised: Experimental results (see the Appendix) show that it is difﬁcult to learn decision rules other than shortcuts through such a learning paradigm.

Original: We believe that the reason is that the excellent representational ability of the deep learning model allows it to represent functionally identical features in different ways.
Revised: We believe that the reason is that the excellent representational ability of the deep learning model allows it to represent functionally identical features in different ways.

Original: Therefore, a simpler and more effective approach is our roadblocks, which forces the model to learn new features by temporarily disabling the already learned features by modifying the input.
Revised: Therefore, a simpler and more effective approach is our roadblocks, which forces the model to learn new features by temporarily disabling the already learned features by modifying the input.

Original: Rather than some distance between features, we remind that functional differences between features may be more important.
Revised: Rather than some distance between features, we remind that functional differences between features may be more important.

Original: From comprehensiveness to exactness.
Revised: From comprehensiveness to exactness.

Original: Our method is designed for general image classiﬁcation tasks.
Revised: Our method is designed for general image classiﬁcation tasks.

Original: The priors about the features and the distribution difference between training and testing sets are unknown.
Revised: The priors about the features and the distribution difference between training and testing sets are unknown.

Original: On the one hand, comprehensiveness (i.e., maximizing the use of effective features in the accessible data) is a reasonable goal since such models are the most robust against complex test scenarios.
Revised: On the one hand, comprehensiveness (i.e., maximizing the use of effective features in the accessible data) is a reasonable goal since such models are the most robust against complex test scenarios.

Original: For comprehensiveness, our approach allows the model to temporarily disable shortcuts and learn new effective features as much as possible.
Revised: For comprehensiveness, our approach allows the model to temporarily disable shortcuts and learn new effective features as much as possible.

Original: Its effectiveness and beneﬁts have been described above.
Revised: Its effectiveness and beneﬁts have been described above.

Original: On the other hand, exactness (i.e., only using some speciﬁc features) is a more realistic goal for many tasks.
Revised: On the other hand, exactness (i.e., only using some speciﬁc features) is a more realistic goal for many tasks.

Original: Our approach also provides a new implementation for feature screening.
Revised: Our approach also provides a new implementation for feature screening.

Original: By ﬁrst training a model to learn features that are not required for the task and then restricting the use of such features, it will be easier for the model to follow the designer’s intention.
Revised: By ﬁrst training a model to learn features that are not required for the task and then restricting the use of such features, it will be easier for the model to follow the designer’s intention.

Original: Achieving the requirement for exactness at the lowest cost will be a key point for the designer.
Revised: Achieving the requirement for exactness at the lowest cost will be a key point for the designer.

Original:  Conclusion
Revised:  Conclusion

Original: In this work, we develop a novel framework to prevent deep learning models from merely using shortcuts (simple decision rules) while ignoring other effective features.
Revised: In this work, we develop a novel framework to prevent deep learning models from merely using shortcuts (simple decision rules) while ignoring other effective features.

Original: Experimental results on multi-attribute datasets in complex testing scenarios demonstrate that our roadblocks have a strong capacity of reducing reliance on shortcuts and learning new knowledge, which is very difﬁcult for conventional deep learning methods and debiasing models.
Revised: Experimental results on multi-attribute datasets in complex testing scenarios demonstrate that our roadblocks have a strong capacity of reducing reliance on shortcuts and learning new knowledge, which is very difﬁcult for conventional deep learning methods and debiasing models.

Original: Extensive experiments also show that roadblocks achieve excellent performance in terms of extracting new knowledge and improving model diversity.
Revised: Extensive experiments also show that roadblocks achieve excellent performance in terms of extracting new knowledge and improving model diversity.

Original: We expect our roadblocks can help shed light on the nature of shortcuts learning in neural networks, and provide a feasible paradigm for training models without depending on shortcuts.
Revised: We expect our roadblocks can help shed light on the nature of shortcuts learning in neural networks, and provide a feasible paradigm for training models without depending on shortcuts.

Original:  Acknowledgments
Revised:  Acknowledgments

Original: This work was supported by the Anhui Provincial Natural Science Foundation under Grant 2108085UD12, and in part by the National Natural Science Foundation of China under Grants
Revised: This work was supported by the Anhui Provincial Natural Science Foundation under Grant 2108085UD12, and in part by the National Natural Science Foundation of China under Grants

Original: U19B2044 and 61836011.
Revised: U19B2044 and 61836011.

Original: We acknowledge the support of GPU cluster built by MCC Lab of
Revised: We acknowledge the support of GPU cluster built by MCC Lab of

Original: Information Science and Technology Institution, USTC.
Revised: Information Science and Technology Institution, USTC.

Original: A Appendix A.1 Implementation Details

Original: Our code is mainly based on PyTorch and will be available at https://github.com/ badcode-iip/Roadblocks .

Original: The datasets used in the paper are all public datasets.

Original: There are many versions of CMNIST, the one we use is the version as LFF [23].

Original: It contains identically distributed training and validation sets, and an unbiased testing set.

Original: We did not use any preprocessing for CMNIST.

Original: The details and download methods of CelebA [20] and BAR [23] can be found in the corresponding papers.

Original: We preprocessed CelebA and BAR (resize, normalize) for easier training, we also applied horizontal ﬂipping for augmentation.

Original: The implementation of t-SNE is from scikit-learn, and the implementations of Grad-CAM and SVCCA are all from open source code.

Original: For Table 1, we use 3-layer MLPs, the number of neurons are (3*28*28, 100, 100, 32, 10).

Original: For the rest of the experiments, we use ResNet-18 as the classiﬁer, and the pre-training parameters provided by PyTorch only in experiments on CelebA and BAR.

Original: The autoencoder is consist of a encoder (3 Conv2d layer) and a correspond decoder.

Original: We use Adam as the optimizer, learning rate is 1e-3 and alpha is 1e-6.

Original: The max epoch is 100.

Original: The batch size is 64 for CMNIST and 16 for CelebA and BAR.

Original: [ λ ac , λ rb , λ rec ] are set to [1, 1, 10] for CelebA and [1, 0.01, 10] for CMNIST and BAR.

Original: A.2 Functionally identical features may have different representations.

Original: We use negative correlation learning to train models with feature-level differences, hoping to learn different decision rules.

Original: In order to show the results more intuitively, we divide the testing set into independently-identical-distribution (i.i.d.) part and out-of-distribution (o.o.d.)

Original: part.

Original: The results are shown in Table 3.

Original: It can be seen that none of them can classify o.o.d.

Original: samples correctly, which means that none of them can learn features other than color.

Original: [Table]

Original: A.3 What if autoencoder is not strong enough?

Original: The reconstruction capability of AE has a direct impact on the setting of roadblocks.

Original: In this section we use 3-layer MLPs to build AE.

Original: By adjusting the reconstruction ability and the weight of reconstruction, we conduct further analysis on the behavior of the model.

Original: While keeping other hyperparameters unchanged, we modiﬁed the number of neurons in the code layer of AE to limit its reconstruction capability, then we repeated the previous experiment, the ﬂuctuation of performance is shown in

Original: Figure 7a.

Original: When the reconstruction capability of AE is low, the model cannot complete the basic i.i.d.

Original: task, and when it is high, the model may again prefer shortcuts.

Original: Only the results in the middle section just meet our expectations.

Original: Fortunately, we can avoid the ﬁrst case by requiring reasonable training accuracy.

Original: And for the second case, we propose two simple strategies, on the one hand, we can consider the setting that just avoided the ﬁrst case; on the other hand, adjusting the weights of reconstruction loss can also guide models with high reconstruction capabilities to make reasonable trade-offs.

Original: Let us change a perspective and discuss the behavior of AE.

Original: It can be seen that with the expansion of the code layer, the accuracy on the i.i.d.

Original: testing set continues to improve, and the accuracy on the o.o.d.

Original: testing set increases ﬁrst and then decreases.

Original: When the reconstruction capability is insufﬁcient,

Original: AE tends to use attributes other than shortcuts, which is understandable because the roadblock loss is preventing AE from using shortcuts.

Original: As the reconstruction capability improves, AE gradually learns shortcuts and weighs the shortcuts and other attributes under the guidance of losses.

Original: When the ability to rebuild is greatly compromised, the weight of reconstruction loss also has impact on the training process.

Original: We use hold the structure of BM and EM as ResNet-18, the weights of

Original: [Figure]

Original: other losses λ rb and λ ac are both set to 1.

Original: The structure of AE is MLP, (512, 256, 512).

Original: When other hyperparameters remain unchanged, the experimental results of different λ rec are shown in Figure 7b.

Original: The settings of λ rec can be roughly divided into three types according to i.i.d.

Original: set accuracy and o.o.d.

Original: set accuracy.

Original: (1) too high, Acc.

Original: i.i.d. is close to 1, and Acc. o.o.d.

Original: is close to 0, which means that AE tends to completely reconstruct original images, and EM can still learn shortcuts since roadblocks are invalid; (2) too low, Acc.

Original: i.i.d. is low, and Acc. o.o.d.

Original: is close to a random result, although roadblocks can work at this time, it is difﬁcult for EM to learn with reconstructed images; (3) appropriately,

Original: Acc.

Original: i.i.d. is good enough, and Acc. o.o.d.

Original: is close to a random result, which indicates that EM has learned attributes other than shortcuts, and no longer relies too much on shortcuts while maintaining accuracy.

Original: It can be seen that when λ rec is not too high, Acc. o.o.d.

Original: is stable, so even if we may not be able to obtain o.o.d.

Original: testing sets in real scenarios, it is not difﬁcult to ﬁnd suitable λ rec under the guidance of Acc.

Original: i.i.d. .

Original: A.4 Can adversarial training handle shortcuts?

Original: We discuss the relationship of our method to adversarial training in the main text.

Original: Table 4 is the performance of simple adversarial training with fast gradient sign untargeted adversarial attack (FGSM)

Original: [6] under the l2 constraint on CMNIST.

Original: The maximum norm of the adversarial perturbation (cid:15) is 0.3, we use Adam as the optimizer, learning rate is 1e-2.

Original: It can be seen that adversarial training does not have a signiﬁcant effect on biased datasets.

Original: [Table]

Original: We further conducted experiments under different adversarial training parameter settings, and the results are shown in Table 5.

Original: It can be seen that the accuracy of the adversarial trained models on the adversarial samples and the clean samples are basically the same.

Original: This is a demonstration that adversarial training has helped the model achieve a certain degree of adversarial robustness.

Original: As epsilon increases, the accuracy of l2 improves, while the accuracy of linf increases ﬁrst, but it is difﬁcult to converge when epsilon is too large.

Original: Some visualizations of adversarial examples shown in Figure 8 can help us understand why adversarial training has limited effect on the shortcut problem.

Original: It can be seen that the difference between the adversarial sample and the real sample is almost invisible to the naked eye, and it is difﬁcult to observe clear digital information in the adversarial noise magniﬁed by 100 times.

Original: We also conduct experiments on project gradient descent (PGD) [21], but it seems difﬁcult to obtain effective adversarial examples using MLPs.

Original: So we use ResNet-18 as the backbone.

Original: epsilon=0.03, k=4, alpha=0.03/4.

Original: The accuracy is 92.91 on CMNIST (5pct) and 88.54 on adversarial examples.

Original: [Table]

Original: [Figure]

Original: (vanilla can achieve 93.34).

Original: Figure 9 shows some samples of PGD.

Original: It can be seen that the noise of PGD seems to be more uniform compared to that of FGSM.

Original: Neither can distinguish colors or numbers with the naked eye.

Original: Based on these experimental results, we believe that simple adversarial training cannot solve the shortcut problem that this paper attempts to solve.

Original: [Figure]

Original: A.5 How to choose hyperparameters?

Original: The weight of loss function is a group of important hyperparameters.

Original: Based on empirical values, we adjust the loss function curve during the experiment to ensure that several objectives can be optimized.

Original: We tested different combinations of lambdas on the BAR dataset and selected the one that performed better, the result is shown in Figure 10.

Original: We refer to LDD’s settings [15] for optimizer.

Original: A.6 More visualization results

Original: We show more visualization experiments to demonstrate the generality of our conclusions.

Original: Figureis the result of CelebA and Figure 12 is the result of BAR.

Original: For both of them, from top to bottom, is

Original: 1e1e 1e 1e 1e3 rec 1e1e- 1e1e 1e r b 59.

Original: the original image, the reconstructed image, the difference, the saliency map of the original model, and the saliency map of EM.

Original: We also show some selected results to try to analyze why our method can improve the generalization ability of the model.

Original: Figure 13 is selected based on the predictions of BM and EM.

Original: Green boxes indicate that the samples are classiﬁed correctly, and red boxes indicate the wrong predictions.

Original: For action classiﬁcation, what should really be concerned with is the action of the person itself rather than other things in the scene.

Original: Our method is able to better focus on the person, which results in better performance on the testing set where the environment changes.

Original: For example, for the vanilla model, mountain climbing is misclassiﬁed as ﬁshing based on climbing ropes, and diving is misclassiﬁed as throwing based on spectator background, while EM can give the correct predictions.

Original: In the proposed approach, AE and EM are alternately trained.

Original: As the training continues, AE may modify the samples in different ways, as shown in Figure 14 image below.

Original: The upper left corner is the original image, followed by the reconstruction results of AE at 10, 20, and 30 epochs.

Original: Climbing

Original: Diving

Original: Fishing

Original: Climbing

Original: Diving

Original: Fishing Climbing

Original: Diving

Original: Fishing

Original: Climbing

Original: Diving

Original: Fishing

Original: Climbing

Original: Diving

Original: Fishing

Original: Fishing

Original: Throwing

Original: Diving Climbing

Original: Diving

Original: Fishing

Original: Fishing

Original: Throwing

Original: Diving

Original: A.7 More datasets results

Original: We selected 10 categories of ImageNet to test the proposed approach in general scenarios.

Original: Thecategories are n01608432, n01641577, n02106166, n02127052, n02389026, n02422106, n02690373, n03095699, n03100240, n03417042.

Original: Figure 15 presents the corresponding reconstruction results and saliency maps.

Original: [Table]

Original: In the ImageNet-9 dataset, we chose to modify the background to generate a challenging testing set (mixed-rand, mixed-next), and the results are shown in Table 7 and Figure 16.

Original: [Table]