C AUSAL A DV : A DVERSARIAL R OBUSTNESS THROUGH THE L ENS OF C AUSALITY
A BSTRACT
The adversarial vulnerability of deep neural networks has attracted signiﬁcant attention in machine learning.
As causal reasoning has an instinct for modeling distribution change, it is essential to incorporate causality into analyzing this speciﬁc type of distribution change induced by adversarial attacks.
However, causal formulations of the intuition of adversarial attacks and the development of robust
DNNs are still lacking in the literature.
To bridge this gap, we construct a causal graph to model the generation process of adversarial examples and deﬁne the adversarial distribution to formalize the intuition of adversarial attacks.
From the causal perspective, we study the distinction between the natural and adversarial distribution and conclude that the origin of adversarial vulnerability is the focus of models on spurious correlations.
Inspired by the causal understanding, we propose the Causal -inspired Adv ersarial distribution alignment method, CausalAdv, to eliminate the difference between natural and adversarial distributions by considering spurious correlations.
Extensive experiments demonstrate the efﬁcacy of the proposed method.
Our work is the ﬁrst attempt towards using causality to understand and mitigate the adversarial vulnerability.
I NTRODUCTION
The seminal work (Szegedy et al., 2014; Biggio et al., 2013) shows that DNNs are vulnerable to adversarial examples, which consist of malicious perturbations imperceptible to humans yet fooling state-of-the-art models (Krizhevsky et al., 2012; Szegedy et al., 2015; Simonyan & Zisserman, 2015;
He et al., 2016).
The lack of robustness hinders the applications of DNNs to some safetycritical areas such as automatic driving (Tuncali et al., 2018) and healthcare (Finlayson et al., 2019).
Therefore, mitigating the adversarial vulnerability is critical to the further development of DNNs.
Human cognitive systems are immune to the distribution change induced by adversarial attacks because humans are more sensitive to causal relations than statistical associations (Gopnik et al., 2004).
Using causal language, causal reasoning can identify causal relation and ignore nuisance factors, i.e., not the cause of labels, by intervention (Pearl, 2009; Peters et al., 2017).
As adversarial perturbations are usually imperceptible and make no impact on human decisions (Szegedy et al., 2014; Goodfellow et al., 2015), it is reasonable to assume that the difference between natural and adversarial examples comes from nuisance factors.
This is because if task-relevant factors of some samples are changed, these samples will make both humans and DNNs change their decisions.
From a causal viewpoint (Zhang et al., 2020a), adversarial attacks can be regarded as a speciﬁc type of distribution change resulting from the intervention on the natural data distribution.
In summary, causal reasoning has the instinct for analyzing the effect of the intervention caused by adversarial attacks, so it is essential to leverage causality to understand and mitigate the adversarial vulnerability.
However, there are two signiﬁcant problems to overcome before using causality to understand and mitigate the adversarial vulnerability.
Firstly, constructing a causal graph is arguably the fundamental premise for causal reasoning (Pearl, 2009; Peters et al., 2017), but how to construct causal graphs in the context of adversarial attacks is still lacking in the literature.
Secondly, using causal language to formalize the intuition of adversarial attacks is the key to connect causality and adversarial vulnerability, but it also remains to be solved.
These two problems are fundamental obstacles, which prevent us from employing causality to contribute to adversarial learning.
To address these challenges, we ﬁrst construct a causal graph to model the perceived data generation process where nuisance factors are considered.
The constructed causal graph can entail a speciﬁc intervention distribution, i.e., the adversarial distribution.
Moreover, the causal graph immediately shows that, given inputs, labels are statistically correlated with nuisance factors, which have no cause-effect to labels.
The spurious correlation implies that if DNNs ﬁt the conditional association between labels and nuisance factors, their performance on different conditional associations between labels and nuisance factors will change accordingly.
Through investigating the distinction between these two distributions induced by nuisance factors, we conclude that adversarial distributions can be obtained by exploiting conditional associations between labels and nuisance factors, where the conditional association of adversarial distributions is drastically different from that of natural distributions.
Namely, the origin of adversarial vulnerability is the focus of DNNs on the spurious correlation between labels and nuisance factors.
According to the causal perspective, an adversarial distribution is crafted by exploiting a speciﬁc conditional association between labels and nuisance factors, this association is drastically different from that of natural distributions.
Intuitively, eliminating the difference in such conditional associations between natural and adversarial distributions can promote the performance of models on adversarial distributions.
Thus, we propose the Causal -inspired Adv ersarial distribution alignment method, CausalAdv, to eliminate the difference between these two distributions.
Surprisingly, we ﬁnd that the proposed method shares the same spirits to existing adversarial training (Goodfellow et al., 2015) variants, i.e., Madry (Madry et al., 2018) and TRADES (Zhang et al., 2019).
We validate the efﬁcacy of the proposed method on MNIST, CIFAR10, and CIFAR100 (Krizhevsky et al., 2009) datasets under various adversarial attacks such as FGSM (Goodfellow et al., 2015), PGD (Madry et al., 2018), CW attack (Carlini & Wagner, 2017), and AutoAttack (Croce & Hein, 2020).
Extensive experiments demonstrate that CausalAdv can improve the adversarial robustness signiﬁcantly.
Our main contributions are:
• We provide a causal perspective to understand and mitigate the adversarial vulnerability, which is the ﬁrst attempt towards using causality to contribute adversarial learning.
• To leverage causality to contribute adversarial learning, we solve two fundamental problems.
Speciﬁcally, we construct a causal graph to model the adversarial data generation process and deﬁne the adversarial distribution to formalize adversarial attacks.
• A defense method called causal-inspired adversarial distribution alignment, CausalAdv, is proposed to reduce adversarial vulnerability by eliminating the difference between adversarial distribution and natural distribution.
Extensive experiments demonstrate that the proposed method can signiﬁcantly improve the adversarial robustness.
A CAUSAL VIEW ON ADVERSARIAL DATA GENERATION
The ability of humans to perform causal reasoning is arguably an essential factor that makes human learning different from deep learning (Sch¨olkopf et al., 2021; Zhang et al., 2020a; Gopnik et al., 2004).
The superiority of causal reasoning endows humans with the ability to identify causal relations and ignore nuisance factors that are not relevant to the task.
In contrast, DNNs are usually trained to ﬁt the perceived information overlooking the ability to distinguish causal relations and statistical associations.
This shortcut solution could lead to overﬁtting to these nuisance factors, which would further result in the sensitivity of DNNs to such factors.
Therefore, we propose incorporating causal reasoning to mitigate the sensitivity of DNNs to these nuisance factors.
Before using causal reasoning to analyze adversarial vulnerability, we need to construct a causal graph, as causal graphs are the key for formalizing causal reasoning (Peters et al., 2017).
In the context of adversarial learning, we desire the causal graph by which both the natural and the adversarial distributions can be generated.
In addition, the graph is required to reﬂect the impact of nuisance factors on these two distributions, so that we can investigate the difference in nuisance factors between these two distributions.
Consequently, we can formally establish the connection between nuisance factors and adversarial vulnerability.
Therefore, we propose constructing a causal graph to model the adversarial generation process where the nuisance factors are considered.
One approach is to use causal structure learning to infer causal graphs (Pearl, 2009; Peters et al., 2017), but it is challenging to apply this kind of approach to high-dimensional data.
Using external knowledge to construct causal graphs is another approach (Zhang et al., 2013; Tang et al., 2020; Sch¨olkopf et al., 2021).
As automatically learning a precise causal graph is out of scope for this work, external human knowledge about the data generation process is employed to construct the causal graph.
[Figure]
observable where C, S, X, Y, E, ˜ X, θ are content variable, style variable, natural data label, per turbation perturbed data and of a neural network respectively.
Speciﬁcally, we construct a causal graph G to formalize the image data generation process using the following knowledge for analyzing adversarial vulnerability.
As there might be a number of different causes of natural data X , we propose to divide all the causes into two categories for simplicity.
We group content-related causes into one category, called content variable C .
The rest causes, i.e., nuisance factors, are grouped into another category, called style variable S , which is content-independent, i.e., S ⫫
C .
This implies that C
→ X ← S .
It is noteworthy that, in this paper, we assume that only the content variable is relevant for the task we care about, i.e.,
C → Y .
Perceived data ˜ X are usually composed of perturbations E and natural data X .
When the perturbation E is designed carefully to fool DNNs,
E should be a compound result of X (the object to be perturbed), Y (the reference for the perturbation), and θ (the targets affected by the perturbation), e.g., white-box attacks (Goodfellow et al., 2015; Carlini & Wagner, 2017; Moosavi-Dezfooli et al., 2016), which means that ( X, Y, θ ) → E 1 .
Leverage all this background knowledge, we obtain the causal graph G formalizing the perturbed data generation process, depicted in Fig.
Based on the causal graph, we can deﬁne valid interventions (Pearl, 2009;
Sch ¨ olkopf et al., 2021) and the corresponding intervention distributions.
Deﬁning valid interventions is equivalent to determining which variables or mechanisms in the causal graph can be intervened.
In this work, we consider both hard and soft interventions (Eberhardt & Scheines, 2007; Correa & Bareinboim, 2020) on the perturbation variable E .
Speciﬁcally, we can use a structural causal model to represent the generating mechanism of E :
[Equation]
where the exogenous variable U E stands for other indeterminacies, e.g., random start noise used in
PGD attack (Madry et al., 2018).
By intervening E in different ways, we can obtain different intervention distributions, which could be either natural or adversarial, over the observed variables.
For example, if we do a hard intervention on E , i.e., do ( E = 0 ) in the graph, the generated data distribution corresponds to the natural distribution P ( X, Y ) .
Different perturbations can be obtained by performing soft interventions on
E , i.e., modifying M .
In the context of adversarial attacks, adversaries aim to maximize a certain objective function (cid:96) ( ⋅ ) to mislead a target model h ( X ; θ ) by searching for the worst perturbation for each instance (Goodfellow et al., 2015; Carlini & Wagner, 2017; Dong et al., 2018; Madry et al., 2018), where θ stands for parameters of the target model.
To formalize the intuition of adversarial attacks, we can ﬁrst search for the adversarial perturbation E adv , and then use a function to approximate the mechanism for generating E adv .
The adversarial perturbation E adv can be obtained by maximizing (cid:96) ( ⋅ ) :
[Equation]
where B is a set of valid perturbations, and the adversarial perturbation E adv is the result of the mechanism M adv , i.e., E adv ≔ M adv ( X, Y, θ , U E ) .
The intervention distribution that corresponds to the adversarial mechanism M adv is deﬁned as the adversarial distribution P θ ( ˜ X, Y ) , where the subscript θ indicates that P θ ( ˜ X, Y ) is crafted to attack the target model h
( X ; θ ) with parameter θ .
M ETHOD
To understand the adversarial vulnerability, we study the difference between the natural and adversarial distributions and conclude that conditional associations between labels and style factors play a crucial role in adversarial vulnerability.
Inspired by the conclusion, we propose a method to eliminate the difference in such conditional associations to mitigate adversarial vulnerability.
3. O RIGIN OF ADVERSARIAL VULNERABILITY
Inspired by the causal graph, we derive a causal understanding of adversarial vulnerability.
According to the causal graph G depicted in Fig.
1, there is a path, S → X ← C → Y , from the style variable S to the label Y when X is given 2 , which leads to the correlation between labels and style variables.
The spurious correlation implies that DNNs can perform well on the training set by ﬁtting the statistical association between Y and S , even though the genuine content information is dropped.
Moreover, if the conditional association between Y and S on the (natural) test set is similar to that on the training set, ﬁtting these spurious correlations will also perform well on the test set.
This is consistent with the recent work (Ilyas et al., 2019), which shows that training DNNs with incorrectly labeled data, i.e., the genuine content information is not utilized for training, yields good accuracy on the (natural) test set.
However, suppose DNNs learn such spurious correlation.
In that case, their performance will change with the conditional association between Y and S .
Consequently, ﬁtting such spurious correlation gives adversaries a chance to fool DNNs.
Therefore, the conditional association between labels and style variables is a key to understand adversarial vulnerability.
To identify the origin of adversarial vulnerability from the spurious correlation perspective, we study the distinction between the adversarial distribution and the natural distribution.
To look at what makes the adversarial distribution different from the natural distribution, we expand the natural
P ( Y ∣ X ) and the adversarial distribution P θ ( Y ∣ ˜ X
[Equation]
where we assume the set of valid styles S is discrete, and s stands for a certain style.
We can see that the difference between P ( Y ∣ X ) and P θ ( Y ∣ ˜ X ) results from two terms, i.e., P θ ( s ∣ ˜ X ) and
P θ ( Y ∣ ˜ X, s ) .
P θ
( s ∣ ˜ X ) represents the change of style information, e.g., textures (Geirhos et al., 2018), transformations (Chen et al., 2020; Mitrovic et al., 2020; He et al., 2020), and domain shifts (Ganin et al., 2016).
It is shown that changing image textures has a signiﬁcant impact on predictions of DNNs (Geirhos et al., 2018), but such drastic style changes will hardly appear in the context of adversarial attacks as perturbations are required to be imperceptible 3 .
Although modifying style variables S is not allowed, adversaries can exploit the conditional association between Y and S , i.e.,
P θ ( Y ∣ ˜ X, s ) , via injecting a speciﬁc perturbation to generate adversarial distributions.
Speciﬁcally, under an appropriate soft intervention, i.e., M adv , the conditional distribution P θ ( Y ∣ ˜ X, s ) can be drastically different from P ( Y ∣ X, s ) .
Namely, the property of adversarial distribution essentially results from the drastic difference in the statistical association between labels and style variables.
Intuitively, if DNNs ﬁt the conditional associations between Y and S , their performance will change
In this work, we focus on imperceptible adversarial examples, i.e. the widely studied (cid:96) p -norm bounded adversarial examples, where drastic style changes hardly appear.
with the spurious correlation.
Consequently, a drastic difference between the conditional association between Y and S will cause signiﬁcant performance degradation of DNNs.
Hence, the origin of adversarial vulnerability is the excessive focus of DNNs on spurious correlations between labels and style variables.
This conclusion provides a causal perspective for the empirical observation, i.e. some features are useful but not robust (Ilyas et al., 2019), so adversarial examples can be viewed as a model phenomenon rather than merely a human phenomenon.
3. T HE ADVERSARIAL DISTRIBUTION ALIGNMENT METHOD
In light of the causal understanding of the adversarial vulnerability, improving adversarial robustness requires forcing DNNs to ﬁt the causal relations rather than merely the statistical associations.
However, only the perceived data can be observed in practice, and the supervised information of content variables is usually unavailable.
Therefore, an approach that can avoid reliance on such supervised information is required.
To develop such an approach, we revisit the difference between the adversarial and natural distributions.
Intuitively, if the difference between the natural and the adversarial distribution is negligible, the adversarial vulnerability of DNNs can be mitigated, as DNNs perform well on the natural distribution (Krizhevsky et al., 2012;
He et al., 2016; Zhang et al., 2021).
Consequently, a straightforward solution for improving adversarial robustness is to align these two distributions.
The aforementioned analysis shows that the property of adversarial distributions to fool DNNs comes from speciﬁc conditional associations between labels Y and style variables S , i.e., P θ ( Y ∣ ˜ X, s ) and P ( Y ∣ X, s ) .
Inspired by the conclusion, we propose an adversarial distribution alignment method to eliminate the difference between the natural and adversarial distributions.
Speciﬁcally, we regard the natural distribution P ( Y ∣ X, s ) as an anchor , then align the adversarial distribution P θ ( Y ∣ ˜ X, s ) with the anchor such that the difference between these two distributions is negligible.
In addition, the relationship between Y and X should also be the same on both the adversarial and natural distributions.
Concretely, we operationalize the intuition of the adversarial distribution alignment by:
[Equation]
Here d ( ⋅ ) is a metric reﬂecting the divergence between two distributions, and λ > 0 presents the weighting of the misalignment penalty.
In addition, we assume X and ˜ X have the same support set because adversarial perturbations are usually bounded.
Solving the adversarial distribution alignment objective is equivalent to search a classiﬁer h ( ˜ X ; θ ) such that the adversarial distribution of the classiﬁer is similar to the natural distribution.
Beneﬁting from the consistency of these two distributions, classiﬁer h ( ˜ X ; θ ) can perform well on both the natural and adversarial distributions.
In practice, the adversarial distribution often cannot be obtained analytically, so we relax the distribution divergence in Eq.
4 to the sum of two divergences, see Appendix A for details.
For the divergence between P ( Y ∣ X ) and P θ ( Y ∣ ˜ X ) , we have:
[Equation]
where KL is the Kullback-Leibler divergence, CE is the cross-entropy loss, Q θ ( Y ∣ X ) is the conditional distribution speciﬁed by the classiﬁer h
( X ; θ ) , and γ is a tunable hyperparameter.
Because adversarial examples are usually generated by adding perturbation to their corresponding natural samples rather than sampled from the adversarial distribution independently, we use X + E adv to approximate examples independently sampled from the adversarial distribution, where X is sampled from the natural distribution independently.
Similar to Eq.
5, the divergence between P ( Y ∣ ˜ X, s ) and P θ ( Y ∣ ˜ X, s ) can be approximated by:
[Equation]
where g ( s ( X ) ; W g ) is a function used for modeling the statistically conditional association between labels and style variables, s ( X ) stands for the integrated representation of X and s , and β is a tunable hyperparameter.
Thus, the overall objective of the proposed adversarial distribution alignment method can be expressed as
[Equation]
To make sure that introducing g can beneﬁt learning h , it is necessary to design an approach to connect model g and model h .
In this paper, we connect g and h by representation sharing.
Interestingly, according to Eq. 4 and Eq. 7, if we omit the spurious correlation between labels and style variables, i.e., λ = 0 , and set γ = 0 , Eq. 7 then becomes the objective function introduced by Madry (Madry et al., 2018).
This suggests that the proposed adversarial distribution alignment method is consistent with the seminal variant Madry (Madry et al., 2018) of adversarial training.
3. R EALIZATION OF ADVERSARIAL DISTRIBUTION ALIGNMENT METHOD
According to Eq. 7, realizing the proposed adversarial distribution alignment method requires predicting labels with all style variables, i.e., modeling the statistically conditional association between
Y and S .
However, there are two major obstacles preventing us from calculating the last term of
Eq. 7.
Speciﬁcally, a) the number of all possible styles s is inﬁnite, so the cost of calculating the expectation in Eq.
7 can grow to inﬁnity; b) the representation of the integrated representation of X and s , i.e., s ( X ) , used for predicting labels is unknown.
To calculate the expectation in Eq. 7, we have to make some assumptions to approximate the distribution of the style variable, as the true distribution is unknown.
Following previous work (Gal & Ghahramani, 2016; Kendall & Gal, 2017), we assume a Gaussian distribution to approximate the unknown distribution.
Speciﬁcally, the representation s ( X ) is estimated by ˆ s ( X ) sampled from a
Gaussian distribution, i.e., ˆ s ( X ) ∼ N ( µ ( X ) , Σ ) , where µ ( X ) is employed to model the Gaussian distribution
’s mean, and Σ is the covariance matrix.
Because little prior knowledge about the covariance matrix is observed, we simply regard the covariance matrix Σ as an identity matrix, i.e., ˆ s ( X ) ∼ N ( µ ( X ) , σ 2 I ) .
Then, the expectation can be approximated by
[Equation]
Thanks to the Gaussian distribution assumption, we can derive an upper bound of the expectation:
[Equation]
where the probability of the i th category of g ( µ ( X ) ; W g ) is (see Appendix B for details)
[Equation]
Here, we simply set g to a linear function, i.e., W g is a linear mapping.
Instead of calculating all styles ˆ s ( X ) , the conclusion of Eq.
9 and Eq.
10 shows that only the mean and variance of ˆ s ( X ) are required to calculate the expectation in Eq.
In light of Eq. 9 and Eq.
10, the challenge of learning the representation of s ( X ) boils down to estimating the mean of ˆ s ( X ) , i.e., no need to estimate every style representation explicitly.
In addition, the variance can be treated as a hyperparameter.
To estimate the mean, we draw inspiration from the proposed causal graph.
According to the causal graph G , the content and style variables are statistically independent, i.e., C ⫫ S .
Thus, the estimated content ˆ c ( X ) and style ˆ s ( X ) are desired to be independent, i.e., ˆ c ( X ) ⫫ ˆ s ( X ) .
The underlying intuition is that we aim to model the causal relation between the content variable and the style variable and omit the spurious correlation between ˆ c ( X ) and ˆ s ( X ) , because the spurious correlation is not a causal relation, although ˆ c ( X ) and ˆ s ( X ) are not statistically independent.
Following previous work (LeCun et al., 1998; Bengio et al., 2013), we regard DNNs as a combination of representation learning modules and linear classiﬁer modules.
Speciﬁcally, we apply a linear function to the learned representation for approximating the content which is further used to predicting labels.
Speciﬁcally, we have ˆ c ( X ) =
ˆ c ( r ( X ; W r ) ; W c ) =
W c r ( X ; W r ) .
That is, W c are parameters used for predicting labels.
According to the Gaussian distribution assumption, we can obtain the estimated style by the reparameterization trick, i.e., ˆ s ( X ) = µ ( X ; W s ) + σ n , where W s presents parameters for modeling the mean, and n is sampled from a normal distribution.
For simplicity, we assume that µ ( X ; W s ) is an afﬁne mapping applied to the learned representation 4 , i.e., µ ( X ; W s )
=
µ ( r ( X ; W r ) ; W s )
=
W s r ( X ; W r ) .
Assume that r
( X ; W r ) is a Gaussian distribution with a covariance matrix M .
Then, the independence ˆ c ( X ) ⫫
ˆ s ( X ) holds if we set W s as an instantiation of the orthogonal complement of W c , i.e., ker ( W c ) ⊥ ⊥ M ker ( W s ) ⊥ .
Here, we deﬁne ⟨ a, b ⟩ M = ⟨ a, Mb ⟩ , and the orthogonality of two subspaces is deﬁned likewise.
Therefore, we can simply employ the learned representation r ( X ; W r ) and the parameters used for predicting labels, i.e., W c , to estimate µ ( X ) , see Appendix C for details.
Combining Eq.
7 and Eq.
9, we derive a realization of adversarial distribution alignment method:
[Equation]
Given X , Eq.
11 encourages the statistically conditional association between labels and style variables of the adversarial distribution to be close to that of the natural distribution.
The explicit distribution alignment can reduce the difference between the adversarial distribution and the natural distribution.
If the conditional association of the adversarial distribution is similar to that of the natural distribution, it should be hard for the adversary to ﬁnd adversarial examples, which is consistent with recent work (Kilbertus et al., 2018;
Sch ¨ olkopf et al., 2021).
E XPERIMENT 4. S ETUPS
Baseline methods.
Our experiments are designed to demonstrate the necessity of considering the spurious correlation between labels Y and style variables S when developing robust models.
Eq.shows that if we set the hyperparameter γ = 0 , and omit the spurious correlation between Y and
S , the proposed method is equivalent to the adversarial training variant Madry (Madry et al., 2018).
Thus, to demonstrate the necessity of considering the spurious correlation, we set γ = 0 and λ >in Eq. 7, named CausalAdv-M, and compare the adversarial robustness of CausalAdv-M with that of Madry.
In addition, the model capacity is often insufﬁcient in adversarial training (Madry et al., 2018), so replacing one-hot labels of the ﬁrst term in Eq.
7 with soft targets, i.e., the model prediction h ( X ; θ ) , can relieve the problem of insufﬁcient model capacity 5 .
Considering the insufﬁcient model capacity, we replace Y in the ﬁrst term of Eq.
7 with the model prediction, and the derived method is called
CausalAdv-T. We ﬁnd that CausalAdv-T becomes the objective introduced in TRADES (Zhang et al., 2019) when we omit the spurious correlation, i.e., λ = 0 , which shows that the proposed method also shares the same spirits to TRADES.
Therefore, to demonstrate the importance of the spurious correlation between Y and S , we compare CausalAdv-M and CausalAdv-T with
Madry and TRADES, respectively.
Evaluation metrics and training details.
To evaluate the robustness for different methods, we compute the test accuracy on natural and adversarial examples with (cid:96) ∞ -norm bounded perturbation generated by: FGSM (Goodfellow et al., 2015), PGD (Madry et al., 2018), and C&W (Carlini &
Wagner, 2017) attacks.
The robustness is evaluated on both the best checkpoint model suggested by (Rice et al., 2020) and the last checkpoint model used in (Madry et al., 2018), respectively.
For
MNIST, we use the same CNN architecture as (Carlini & Wagner, 2017; Zhang et al., 2019).
For
CIFAR10 and CIFAR100, two architectures are employed: ResNet-18 (He et al., 2016) and WRN34-10 (Zagoruyko & Komodakis, 2016).
The settings of attacks and hyper-parameters for training are the same as previous works, more details can be found in Appendix D.
4. R OBUSTNESS EVALUATION
We evaluate the robustness of Madry, TRADES, and the proposed method on MNIST, CIFAR10, and
CIFAR100 against various attacks (Goodfellow et al., 2015; Madry et al., 2018; Carlini & Wagner, 2017), which are widely used in the literature.
We report the classiﬁcation accuracy on MNIST in
[Table] [Table]
Table 1, where “Natural” denotes the accuracy on natural test images.
We denote by PGD-40 the
PGD attack with 40 iterations for generating adversarial examples, which also applies to the C&W attack.
The results of ResNet-18 on CIFAR10 and CIFAR100 are illustrated in Table 2 and Table 3, respectively.
The results of WRN-34-10 are in Appendix E.
We can see that the proposed method achieves the best robustness against all three types of attacks, demonstrating that taking into account the spurious correlation can signiﬁcantly improve the adversarial robustness.
To further understand the comparative effects of different terms of the proposed method, we reorganize the robust accuracy of the best checkpoint trained on CIFAR-10 and CIFAR-100 in Appendix F
4. D ISCUSSION
Consideration of gradient obfuscation.
According to the criterion suggested by (Athalye et al., 2018), we exclude the potential effect of gradient obfuscation by showing the following phenomenons:
a) the performance of our method on FSGM attack ( 59 . 17% ) is better than iterative attacks PGD20 ( 51 .
82% ) and C&W-20 ( 50 . 93% ); b) the performance of our method on black-box PGD- ( 83 .
% ) and C&W-20 ( 83 .
57% ) attacks is better than that on white-box attacks ( 51 .
; c) strong attacks cause lower accuracy than weak attacks, i.e., the accuracy on PGD-20 and PGD-100 are 51 .
82% and 48 .
05% , respectively.
In addition, no gradient shattering operator is used in our method.
All these results are evaluated on the CIFAR10 dataset using the last checkpoint of ResNet-18.
Thus, according to the criterion suggested by (Athalye et al., 2018), the robustness improvement of the proposed method does not result from gradient obfuscation.
Consideration of adaptive attack.
According to the adaptive attack criterion (Tramer et al., 2020), we augment the original objective function used in the PGD attack with the proposed adversarial distribution alignment loss to implement adaptive attacks, i.e., Eq. 11.
Under the adaptive attack, the accuracy is 51 .
68% , while under the original PGD-20 attack is 51 .
82% , demonstrating that the proposed method is genuinely robust.
Consideration of AutoAttack.
Following previous work (Pang et al., 2020), we verify the effectiveness of the proposed method on AutoAttack (Croce & Hein, 2020).
According to the commonly used setting, see (Pang et al., 2020), we report robust accuracy of WRN-34-10 trained with CIFAR dataset on Auto-Attack, Madry: 49 .
58% , CausalAdv-M: 51 .
56% , TRADES: 52 .
46% , CausalAdv-
[Table] [Table]
[Table]
Mitigating robust overﬁtting.
The recent work (Rice et al., 2020)
ﬁrst studies the robust overﬁtting phenomenon.
Robust overﬁtting means that further training will increase the robust training accuracy and test accuracy on natural data after a certain training epoch but the robust test accuracy will decrease.
The robust overﬁtting phenomenon of Madry (Madry et al., 2018) is depicted in Fig.
It can be seen that the robust test accuracy of Madry decreases to about 44% , while the best robust accuracy of Madry is 51 .
In contrast, the proposed method drastically reduces the difference between the best robust accuracy 54 .
44% and the robust accuracy 50 .
93% of the last checkpoint.
R ELATED WORK
Adversarial attack.
Unlike the assumption employed in noisy labels (Han et al., 2020;
Liu &
Tao, 2015; Xia et al., 2020), adversarial attacks assume that the input determines noise.
Since the realization of the adversarial example phenomenon (Biggio et al., 2013; Szegedy et al., 2014), tons of adversarial attacks have been proposed (Moosavi-Dezfooli et al., 2016; Goodfellow et al., 2015;
Carlini & Wagner, 2017; Dong et al., 2018;
Tu et al., 2019; Madry et al., 2018; Croce & Hein, 2020).
Among these attacks, FGSM Goodfellow et al.
(2015), PGD attack (Madry et al., 2018)
, C&W attack (Carlini & Wagner, 2017), and Auto-Attack (Croce & Hein, 2020) are the most commonly used attacks for evaluating robustness.
Adversarial defense.
The development of adversarial attacks promotes the progress of adversarial defense and detection.
Recent work on improving adversarial robustness mainly falls into two categories: certiﬁed defense (Raghunathan et al., 2018; Wong & Kolter, 2018; Singla & Feizi, 2020) and empirical defense and detection with two-sample test (Najaﬁ et al., 2019; Carmon et al., 2019;
Shafahi et al., 2019;
Wong et al., 2019; Pang et al., 2020; Rice et al., 2020; Ma et al., 2018; Gao et al., 2021).
Detailed discussions of these exciting works can be found in Appendix G.
Causal reasoning.
The ﬁeld of graphical causality, like machine learning, has a long history, see
Pearl (2009);
Sch ¨ olkopf et al.
; Peters et al.
One purpose of causal reasoning is to pursue the causal effect of interventions, contributing to achieving the desired objectives.
Recent work shows the beneﬁts of introducing causality into machine learning from various aspects (Zhang et al., 2020a;
Mitrovic et al., 2020; Teshima et al., 2020; Tang et al., 2020; Sauer & Geiger, 2020;
Tang et al., 2021).
More details about relevant works can be found in Appendix H.
C ONCLUSION
In this paper, we provide a novel causality viewpoint for understanding and mitigating adversarial vulnerability.
Through constructing the causal graph of the adversarial data generation process and formalizing the intuition of adversarial attacks, we show that the spurious correlation between labels and style variables is important for understanding and mitigating adversarial vulnerability.
Inspired by the observation, we propose the adversarial distribution alignment method, which takes the spurious correlation into account for robustness improvement.
In addition, we ﬁnd that the proposed method shares the same spirits to existing adversarial training variants.
In future work, we will develop more effective algorithms to leverage or eliminate the spurious correlation between labels and style variables to further improve adversarial robustness.
In addition, we will explore the uses of counterfactual statements to explain and mitigate the adversarial vulnerability.
In sum, we make a ﬁrst step towards employing causality to contribute to adversarial learning.
ACKNOWLEDGMENT
We thank the area chair and reviewers for their valuable comments.
YGZ and BH were supported by the RGC ECS No. 22200720 and NSFC YSF No. 62006202.
YGZ and XMT were supported by
NSFC No. 61872329.
MMG was supported by Australian Research Council Project DE210101624.
TLL was supported by Australian Research Council Projects DE-190101473 and DP-220102121.
KZ would like to acknowledge the support by the National Institutes of Health (NIH) under Contract
R01HL159805, by the NSF-Convergence Accelerator Track-D award #2134901, and by the United
States Air Force under Contract No.
FA8650-17-C7715.
E THICS S TATEMENT
This paper does not raise any ethics concerns.
This study does not involve any human subjects, practices to data set releases, potentially harmful insights, methodologies and applications, potential conﬂicts of interest and sponsorship, discrimination/bias/fairness concerns, privacy and security issues, legal compliance, and research integrity issues.
R EPRODUCIBILITY S TATEMENT
To ensure the reproducibility of experimental results, we open source our code https:// github.com/YonggangZhangUSTC/CausalAdv.git .
A D ERIVATION OF DISTRIBUTION DIVERGENCE
Considering that the adversarial distribution often cannot be obtained analytically, we need to relax the distribution divergence in Eq.
4, i.e., d ( P ( Y ∣ X ) , P θ ( Y ∣ ˜ X )) .
Without loss of generality, the metric d in Eq.
4 can be realized as total variation distance (TVD).
Thus, according to the deﬁnition of TVD, we have
[Equation]
where Q θ ( Y ∣ X ) is the conditional distribution speciﬁed by the classiﬁer h
( X ; θ ) .
According to the Pinsker inequality, i.e., d ( P, Q ) ≤
[Equation]
, where KL is the Kullback-Leibler divergence, we have an upper bound of Eq.
[Equation]
Now, we can employ the upper bound Eq. 13 as a surrogate loss.
In practice, we can replace
[Equation]
with KL ( P ∣∣ Q ) .
The intuition is that the difference between
[Equation]
and KL ( P ∣∣ Q ) is relatively small when KL ( P ∣∣ Q ) is not extremely large, so the replacement will not introduce much difference.
Thus, we derive Eq.
5: KL ( P θ ( Y ∣ ˜ X )
∣∣
Q θ ( Y ∣ ˜ X ))
+ γKL ( P ( Y ∣ X )
∣∣
Q θ ( Y ∣ X )) , where γ is a tunable hyperparameter.
Here, we give an intuitive explanation of why optimizing Eq.
5 can make the adversarial distribution similar to the natural distribution.
Considering that both the adversarial distribution P θ ( Y ∣ ˜ X ) and the anchor Q θ ( Y ∣ ˜ X ) are parameterized by θ , so optimizing θ to minimize KL ( P θ ( Y ∣ ˜ X )
∣∣
Q θ ( Y ∣ ˜ X )) can force the adversarial distribution be similar to the distribution speciﬁed by the classiﬁer h
( X ; θ ) , i.e., Q θ ( Y ∣ ˜ X ) .
That is, optimizing Q θ ( Y ∣ ˜ X ) will also change the adversarial distribution P θ ( Y ∣ ˜ X ) .
In addition, minimizing the divergence
KL ( P ( Y ∣ X ) ∣∣
Q θ ( Y ∣ ˜ X )) can endow the classiﬁer with the ability to provide a good prediction performance.
Thus, minimizing KL ( P θ ( Y ∣ ˜ X ) ∣∣
Q θ ( Y ∣ ˜ X )) will make the adversarial distribution be similar with the natural distribution.
To verify the replacement will not introduce much difference, we evaluate the robustness of
CausalAdv-M and CausalAdv-T on the CIFAR-10 dataset with two losses, i.e., Eq. 5 and Eq. 13.
The results evaluated on best checkpoints are shown in Table 4.
[Table]
CausalAdv-T* 79.
B C ALCULATION OF THE EXPECTATION ON THE STYLE INFORMATION
We provide details of calculating E ˆ s ( ˜ X ) ∼ N ( µ ( ˜ X ) ,
σ 2 I ) CE ( g ( ˆ s ( ˜ X ) ; W g ) , Y ) .
We assume a normal distribution for the styles, i.e., ˆ s ( ˜ X ) ∼ N ( µ ( ˜ X ) , σ 2 I ) .
According to the deﬁnition of the crossentropy loss, for a input pair ( x, y ) we have:
[Equation]
where the inequality follows from the Jensen’s inequality: E log ( X ) ≤ log E X
[Equation]
the expectation is calculated by leveraging the moment-generating function:
[Equation]
Note that
[Equation]
we deﬁne the function g ( µ ( x ) ; W g ) for simplicity:
[Equation]
Besides seting g to a linear model
[Table]
non-linear models
[Table]
e.g., neural networks, can also be employed in practice.
To verify the inﬂuence introduced by selecting different instantiation of model g
[Table]
we compare different realizations of mode g , i
[Table]
e., linear mapping and non-linear neural networks.
The following results suggest that our method is relatively robust to the selection of model g .
Note that, all these results are evaluated on the best checkpoint models trained on CIFAR-10 dataset.
[Table]
C R ELATIONSHIP BETWEEN ORTHOGONALITY AND STATISTICAL INDEPENDENCE
We give the proof for the following lemma in Sec. 3.3.
Note that, we use R to present the learned representation of X , and replace X with R for simplicity.
Lemma 1.
R ∈ R
d is the learned representation, where d is the number of dimension of R .
Assume that R is a normal distribution with mean m and covariance matrix M .
The content used for predicting labels, i.e., logits, is obtained by applying a linear functions to R , i.e., ˆ c ( R ) =
W c R , where
W c are parameters used for mapping R to logits.
The style is modeled by a normal distribution,
i.e., ˆ s ( R ) =
µ ( R ; W s ) + Σ ( Y ) 12 n , where W s presents parameters for modeling the mean of styles, and n is sampled from a standard normal distribution.
Assume that µ ( R ; W s ) is a linear function, i.e., ˆ s ( R ) =
W s R + Σ ( Y ) 12 n .
Then, setting W s as an instantiate of the orthogonal complement of W c leads to statistical independence, i.e., ˆ c ( R ) ⫫
ˆ s ( R ) .
Here, ⫫ denotes the statistical independence, and we deﬁne ⟨ a, b ⟩ M = ⟨ a, Mb ⟩ for a given semi-deﬁnite matrix M .
The orthogonality
A ⊥ M B of two subspaces A and B is deﬁned likewise.
Proof.
Under the assumption in Lemma 1, setting W s as an instantiate of the orthogonal complement of W c , we have:
[Equation]
D
M
ORE DETAILS ABOUT EVALUATION METRICS AND TRAINING DETAILS
Evaluation metrics.
For MNIST dataset, we set the maximum perturbation bound (cid:15) = 0 .
3 , perturbation step size η = 0 . 01 , and the number of iterations K = 40 for PGD and C&W attacks, which keeps the same as (Zhang et al., 2019).
Following (Rice et al., 2020), we set perturbation bound (cid:15) = 8 / 255 , perturbation step size η = (cid:15) / 10 , and the number of iterations K = 20 for CIFARdataset.
Training details.
For MNIST, we use the same CNN architecture as (Carlini & Wagner, 2017;
Zhang et al., 2019).
Following (Zhang et al., 2019), the network is trained using SGD with 0.
momentum for 50 epochs with an initial learning rate 0.01, and the batch size is set to 128.
Hyperparameters used to craft adversarial examples for training are the same as those used for evaluation.
These two networks share the same hyper-parameters: we use SGD with 0 .
9 momentum, weight decay 2 × 10 − 4 , batch size 128 , and an initial learning rate of 0 . 1 .
The maximum epoch is 120 , and the learning rate is divided by 10 at epoch 60 and 90, respectively.
To generate adversarial examples for training, we set the maximal perturbation (cid:15) = 8 / 255 , the perturbation step size η = 2 / 255 , and the number of iterations K = 10 , which is the same as (Rice et al., 2020).
In all of our experiments β is set to 1.0.
For CausalAdv-M, λ is set to 1 . 0 and 0 .
5 for CIFAR-10 and CIFAR-100 datasets, respectively.
For CausalAdv-T, λ is set to 0 .
5 and 1 .
for CIFAR-10 and CIFAR-100 datasets, respectively.
E E XPERIMENTS OF WRN-34-10 ON CIFAR-
[Table]
In Table
[Table]
we report the accuracy of WRN-34-10 (Zagoruyko & Komodakis, 2016) of Madry,
TRADES
[Table]
and the proposed method on CIFAR-10 against various attacks,
i
[Table]
e., FGSM, PGD
[Table]
and
C&W attacks, which are widely used in the literature.
Here, “Natural” denotes the accuracy of natural test images.
We denote by PGD-20 the PGD attack with 20 iterations for generating adversarial examples, which also applies to the C&W attack.
We can see that the proposed method achieves the best robustness against all three types of attacks, demonstrating that taking into account the spurious correlation can signiﬁcantly improve the adversarial robustness.
Note that the standard deviations of 5 runs are omitted, because they hardly affect the results.
F A BLATION STUDY
[Table]
We implicitly conducted ablation studies when designing Table 1, Table
[Table]
and Table 3.
To further understand the comparative effects of different terms of the proposed method, we reorganize the robust accuracy of the best checkpoint trained on CIFAR-10 and CIFAR-100 in Table 7.
Comparing
Madry, TRADES
[Table]
and CausalAdv-M, we ﬁnd that introducing the second ( t 2 ) and the third term ( t 3 ) can improve the robustness and that the effect of these two terms is close.
Similarly, comparing
TRADES and CausalAdv-T, we see that introducing the third term ( t 3 ) can further improve the robustness.
To further analyze the superiority of our method, we compare CausalAdv-M with Madry to explore which kind of adversarial examples that CausalAdv-M is more robust to.
These analyses are based on the spurious perspective, as one main conclusion of our paper is that the origin of adversarial vulnerability is the excessive focus of DNNs on spurious correlations between labels and style variables.
Speciﬁcally, we calculate the KL-divergence between KL ( P ( Y ∣ h ( x, s ))
∣∣
P ( Y ∣ h ( x adv , s ))) for each input x , and divide samples into several (10 in our experiment) bins according to the KLdivergence.
Then, we evaluate the robust accuracy of models trained with CausalAdv-M and Madry in each bin sample, and results are depicted in Fig.
We can see that CausalAdv-M is more robust to samples leading to large KL-divergence than Madry.
According to these empirical results, we can conclude that the proposed method is more robust to samples causing a signiﬁcant difference between natural and adversarial distributions.
To further verify that CausalAdv-T can outperform TRADES interms of both the natural and adversairal robustness, we compare the robust accuracy of CausalAdv-T with that of TRADES on
CIFAR-10 and CIFAR-100 dataset.
The results evaluated on the best checkpoint models are shown in Table 8.
We can see that CausalAdv-T can improve both the natural and the adversarial accuracy.
[Table]
To explore the sensitivity of our method on hyperparameters
[Table]
we evaluate our methods on CIFAR and CIFAR100 datasets.
The results are organized in Table 9.
Results in Table 9 demonstrate that
[Table]
both CausalAdv-M and CausalAdv-T are relatively insensitive to the hyperparameters.
G M ORE DETAILS ABOUT ADVERSARIAL LEARNING
Recent work on improving adversarial robustness mainly falls into two categories: certiﬁed defense and empirical methods.
Certiﬁed defense (Raghunathan et al., 2018; Wong & Kolter, 2018; Singla & Feizi, 2020) aims to endow the model with provably adversarial robustness against norm-bounded perturbations.
Although the certiﬁed defense strategy is promising, the empirical defense (Goodfellow et al., 2015;
Madry et al., 2018; Zhang et al., 2019;
Pang et al., 2020; Wong & Kolter, 2018;
Xie et al., 2019;
Yang et al., 2019), especially the adversarial training method (Goodfellow et al., 2015; Madry et al., 2018; Zhang et al., 2019), is currently the most effective strategy.
Empirical defense ﬁrstly generates adversarial examples using a certain adversarial attack, then incorporates the generated adversarial examples into the training process.
Recently, an empirical detection strategy is to utilize a twosample test to detect adversarial examples (Gao et al., 2021).
In the following, we mainly discuss defense strategies, as the detection approach is not the main focus of this paper.
Recently, various efforts (Najaﬁ et al., 2019; Carmon et al., 2019; Shafahi et al., 2019;
Wong et al., 2019; Pang et al., 2020;
Rice et al., 2020) have been devoted to improving adversarial training.
One line of work focuses on accelerating the training procedure (Shafahi et al., 2019; Wong et al., 2019).
Another line of research (Najaﬁ et al., 2019; Carmon et al., 2019) shows a promising direction that unlabeled training data can signiﬁcantly mitigate the adversarial vulnerability.
Lastly, recent work (Pang et al., 2020; Rice et al., 2020) provides an interesting direction where these methods rethink the adversarial training from a exciting aspect, i.e., rethinking the role of normalization (Pang et al.,
and basic training strategies (Rice et al., 2020).
However, all these methods overlook the spurious correlation between labels and the style information.
Another related work is (Ilyas et al., 2019), which provides an interesting viewpoint, i.e., adversarial examples can be viewed as a human phenomenon because the model’s reliance on useful but not robust features leads to adversarial vulnerability.
Our work gives a new causal perspective of adversarial vulnerability.
Speciﬁcally, a) (Ilyas et al., 2019) found some features were useful but not robust, while our work explores the phenomenon’s fundamental cause and provides a clear explanation of why some features are useful but not robust: Given X , labels Y are spuriously correlated with the style variables, so ﬁtting the spurious correlation can predict labels.
Thus, the style variables can be viewed as ‘features’; b) (Ilyas et al., 2019) claimed that adversarial examples could be viewed as a human phenomenon, while our work shows that adversarial examples can be viewed as a model phenomenon rather than merely a human phenomenon.
Speciﬁcally, the adversarial vulnerability results from ﬁtting the correlation between labels and style variables and failing to ﬁt the causal relations, i.e., DNNs fail to extract content variables.
H M ORE DETAILS ABOUT CAUSAL REASONING
The most relevant work is CAMA (Zhang et al., 2020a) that aims to improve the robustness of
DNNs on unseen perturbation via explicitly modeling the perturbation from a causal view.
The main difference between our method and CAMA is that we focus on the adversarial vulnerability while CAMA aims to improve the robustness of unseen perturbations.
In addition, CAMA assumes a hard intervention on a latent variable.
It promotes robustness via modeling the perturbation in the latent space.
In this paper, we employ a soft intervention and propose to penalize DNNs when the adversarial distribution is different from the natural distribution.
A recent work (B ¨ uhlmann, 2020) also aims to connect robust learning and causality, but the main focus of (B ¨ uhlmann, 2020) is on out-of-distribution generalization, which is different from adversarial learning.
Another related work is RELIC (Mitrovic et al., 2020), a regularizer used in self-supervised learning that uses the independence of mechanisms (Peters et al., 2017) and encourages DNNs to be invariant to different augmentations of the same instance.
The self-supervised learning method (Mitrovic et al., 2020) also constructs a causal graph to model the data generation process, but the focus of
RELIC is on the content invariant property, overlooking the importance of style information.
One concurrent work (Tang et al., 2021) propose using the instrumental variable to perform causal intervention, based on a strong assumption that the adversarial vulnerability results from the confounding effect.
In contrast, merely some general assumptions are required in this paper, e.g., causal model assumption.
Although (Sagawa et al., 2020) takes spurious correlations into account, (Sagawa et al., 2020) proposes using prior knowledge to group the training data to avoid the reliance on spurious correlations.
Difference from (Sagawa et al., 2020), our method does not rely on prior knowledge.
Another related work is CORE (Heinze-Deml & Meinshausen, 2021), a regularizer inspired by a causal graph is proposed to minimizing the variance of prediction and loss condition on label and ID information to mitigate the inﬂuence of domain shift.
Different from (Heinze-Deml & Meinshausen, 2021), our method is designed to eliminate the difference between the natural and adversarial distributions.