A C AUSAL V IEW ON R OBUSTNESS OF N EURAL N ETWORKS
Anonymous authors
Paper under double-blind review
A BSTRACT
We present a causal view on the robustness of neural networks against input manipulations, which applies not only to traditional classiﬁcation tasks but also to general measurement data.
Based on this view, we design a deep causal manipulation augmented model (deep CAMA) which explicitly models the manipulations of data as a cause to the observed effect variables.
We further develop data augmentation and test-time ﬁne-tuning methods to improve deep CAMA’s robustness.
When compared with discriminative deep neural networks, our proposed model shows superior robustness against unseen manipulations.
As a by-product, our model achieves disentangled representation which separates the representation of manipulations from those of other latent causes.
I NTRODUCTION
Deep neural networks (DNNs) have great success in many real-life applications, however, they are easily fooled even by a tiny amount of perturbation (Szegedy et al., 2013; Goodfellow et al., 2015;
Carlini & Wagner, 2017b; Athalye et al., 2018).
Lack of robustness hinders the application of DNNs to critical decision making tasks such as uses in health care.
To address this, a deep learning practitioner may suggest training DNNs with datasets that are not only big but also diverse.
Indeed, data augmentation and adversarial training have shown improvements in both the generalization and robustness of DNNs (Kurakin et al., 2016;
Perez & Wang, 2017; Madry et al., 2017).
Unfortunately, this does not address the vulnerability of DNNs for unseen manipulations.
For example, as shown in
Figure 1, a DNN trained on clean MNIST digits fails to classify shifted digits.
Although observing (adversarial) perturbations of clean data in training improves robustness against that particular manipulation (the green line), the DNN is still fragile when unseen manipulations are present (orange line).
Since it is unrealistic to augment the training data towards all possible manipulations that many occur, a principled method that fundamentally improves the robustness is much needed.
On the other hand, humans naturally understand the independent causal mechanisms for visual recognition tasks, where the generative process of the perceived view is composed of modules that do not inﬂuence each other (Parascandolo et al., 2017).
After learning the concept of an “elephant”, a child can identify the elephant in a photo taken under any lightning condition, location, etc.
Importantly, the elephant, the lightning condition, and the location are causes of the presented view in the photo.
Therefore we argue that the incapability for causal reasoning (Pearl & Mackenzie, 2018;
Gopnik et al., 2004) is the reason of DNN’s vulnerability to (adversarial) data manipulations.
This work discusses the robustness of DNNs from a causal perspective.
Our contributions are:
• A causal view on robustness of neural networks.
We argue from a causal perspective that adversarial examples for a model can be generated by manipulations on the effect variables and/or their unseen causes.
Therefore DNN’s vulnerability to adversarial attacks is due to the lack of causal understanding.
• A causal inspired deep generative model.
We design a causal deep generative model which takes into account the unseen manipulations of the effect variables.
Accompanied with this model is a test-time inference method to learn unseen manipulations and thus improve classiﬁcation accuracy on noisy inputs.
Data augmentation techniques can also be safely applied to our model during training without deteriorating its generalization ability to unseen manipulations.
Compared to DNNs, experiments on both MNIST and a measurementbased dataset show that our model is signiﬁcantly more robustness to unseen manipulations.
[Figure]
A C AUSAL V IEW ON R OBUSTNESS OF N EURAL N ETWORKS
Discriminative DNNs are not robust to manipulations such as adversarial noise injection (Goodfellow et al., 2015; Carlini & Wagner, 2017a;
Athalye et al., 2018), rotation and shift.
They do not understand the causal mechanisms of the data generating process
[Figure]
which leads to overﬁting to nuisance factors that are less related to the ground truth classiﬁcation results.
By exploiting the overﬁt to the nuisance factors, an adversary can easily manipulate the inputs to fool discriminative DNNs into predicting the wrong outcomes
[Figure]
On the contrary
[Figure]
we as human can easily recognize an object in a scene and be indifferent to the changes in other aspects such as background, viewing angle, the presence of a sticker to the object, etc.
More importantly, our recognition is not affected even when some of the perturbations, e.g. changes in the lighting condition, are signiﬁcant.
We argue that the main difference here is due to our ability to perform causal reasoning , which identiﬁes independent mechanisms that are not causally related to the object recognition results (Freeman, 1994; Peters et al., 2017; Parascandolo et al., 2017).
This leads to robust human perception to not only a certain type of perturbations, but also to many types of manipulations.
Thus we argue that one should incorporate causal mechanisms into model design, and make the model robust on the level of different types of perturbations.
Before presenting our causally informed model, we ﬁrst deﬁne a valid manipulation of inputs in a causal sense.
A valid manipulation is a perturbation on data, which only changes the effects, not the cause of the target.
We visualize a causal graph in Figure 2, where the arrows indicate the causeeffect relationship between variables.
Take hand-written digit classiﬁcation for example, X is the image of a digit and Y is the class label.
The appearance of X is an effect of the digit number Y , latent causes Z such as writing styles, and possible manipulations M , such as rotation or translation.
Changes to Z and M cause the appearance of X to change, but X still carries the same information about Y regardless of these perturbations, since Z , M and Y are independent mechanisms.
Thus, any manipulation that does not inﬂuence the Y → X relationship are valid manipulations.
Humans are extremely robust to these manipulations while machine learning algorithms are vulnerable.
In summary, from the causal perspective, any manipulation M on data X , that is a co-parent of
Y , is a valid manipulation.
This deﬁnition includes many manipulations used in existing work on the robustness of neural networks, such as noise injection, shift and rotation (Engstrom et al., 2019).
Ideally, a machine learning model should be able to generalize to any valid manipulation, at the same time training with manipulated data of certain types should never harm the model’s robustness to unseen manipulations.
However, discriminative deep learning models ignore the causal structure and consider X → Y only, which explains their vulnerability to data manipulations.
Inspired by causal reasoning of humans, we propose a deep learning framework concerning the causal relationship.
T HE C AUSAL M ANIPULATION A UGMENTED M ODEL
We propose a deep CAusal Manipulation Augmented model (deep CAMA), which takes into account the causal relationship for model design.
Our proposed model is more robust to unseen manipulations on effect variables, and more importantly, our model can learn these manipulations
[Figure]
[Figure]
without supervision.
The robustness can be further improved by training-time data augmentation, without sacriﬁcing the generalization ability to unseen manipulations.
Below we ﬁrst present the deep CAMA for single modality data, which focuses on predicting Y using X , and then present a generic deep CAMA for multimodality measurement data.
3. D EEP CAMA FOR SINGLE MODALITY DATA
The task of predicting Y from X covers a wide range of applications such as image/speech recognition and sentiment analysis.
Normally a discriminative DNN takes X as input and directly predicts (the distribution of) the target variable Y .
Generative classiﬁers, on the other hand, build a generative model Y → X , and use Bayes’ rule for predicting Y given X : p ( y | x )
= p ( y ) p ( x | y ) /p
( x ) .
We design deep CAMA (Figure 3) following the causal relationship as shown in Figure 2.
Taking
MNIST for example: Y is the label and X is the image, Z models the latent style of the digits, and
M handles the manipulations that we desire the model to be robust to.
The model is deﬁned as:
[Equation]
For efﬁcient inference we follow the amortized inference approach in variational auto-encoders (Kingma & Welling, 2013; Rezende et al., 2014; Zhang et al., 2018) and deﬁne an inference network as the approximate posterior distribution:
[Equation]
We use φ to denote all the parameters of the encoder network and φ = { φ 1 , φ 2 } , where φ 1 is the parameter for the encoder network for the variational distribution q φ 1 ( z | x, y, m ) , and φ 2 is used for the q φ 2 ( m | x ) part.
Note that we assume the dependence of M on X only in q φ 2 ( m | x ) , which, as we shall show later, allows deep CAMA to learn unseen manipulations with unlabelled noisy data.
The network architecture is presented in Figure 4.
For the p model, the cause variables Y , Z and
M are ﬁrst transformed into feature vectors h Y , h Z and h M .
Later, these features are merged together and then passed through another neural network to produce the distributional parameters of p θ ( x | y, z, m ) .
For the approximate posterior q , two different networks are used to compute the distributional parameters of q φ 2 ( m | x ) and q φ 1 ( z | x, y, m ) , respectively.
Model training Assume that during training, the model observes clean data D
= { ( x n , y n ) } only.
In this case we set the manipulation variable M to a null value, e.g. do ( m = 0) , and train deep
CAMA by maximizing the likelihood function log p ( x, y | do ( m = 0)) under training data.
Since this marginal distribution is intractable, we instead maximize the intervention evidence lower-bound (ELBO) with do ( m = 0) , i.e. max θ,φ E D [ ELBO ( x, y, do ( m = 0))] , with the ELBO deﬁned as
[Equation]
See appendix A for a detailed derivation. If noisy data D (cid:48) is available during training, then similar to data augmentation and adversarial training (Goodfellow et al., 2015; Tram`er et al., 2018; Madry et al., 2017), we can augment the training data with this noisy data.
We still use the intervention ELBO (3) for clean data.
For the manipulated instances, we can either use the intervention ELBO with do ( m = m 0 ) when the noisy data D (cid:48)
= { ( m 0 ( x ) , y ) } is generated by a known manipulation m 0 , or, as done in our experiments, infer the latent variable M for unknown manipulations.
This is achieved by maximizing the ELBO on the joint distribution log p ( x, y ) using noisy data:
[Equation]
and therefore the total loss function to be maximized is deﬁned as
[Equation]
Our causally consistent model effectively disentangles the latent representation: Z models the unknown causes in the clean data, such as personal writing style; and M models possible manipulations which the model should be robust to, such as shift, rotation, noise etc.
Due to independent mechanism assumptions in causality, the inﬂuence of Y , Z and M on X can be independently applied.
Thus, with our model design, we can also ensure that the dependencies Y → X and Z → X are not affected by noisy data present during training.
As a result, deep CAMA’s can still generalize to unseen manipulations even after seeing lots of noisy datapoints from other manipulations, in contrast to the behavior of discriminative DNNs as shown in Figure 1.
Prediction In general the test data ˜ D can be noisy, and we would like our model to be robust to the unseen manipulated test data.
Thus, at test-time, M is unknown, and deep CAMA classiﬁes an unseen test data x ∗ , using a Monte Carlo approximation to Bayes’ rule with samples m u ∼ q φ 2 ( m | x ) , z [Equation] ∼ [Equation]
[Equation]
In addition, deep CAMA can be adapted to the unseen manipulations present at test time without labels on the noisy data .
From the causal graph, the conditional distributions p ( X | Y ) and p ( X | Z ) are invariant to the interventions on X based on the independent mechanism assumption (Peters et al., 2017), however, we would like to learn the manipulation mechanism M → X .
As shown in
Figure 4, for the generative model, we only ﬁne-tune the networks that are dependent only on M , i.e. NN pM by maximizing the ELBO of the marginal distribution log p ( x ) :
[Equation]
To reduce the possibly negative effect of ﬁne-tuning to model generalization, we use a shallow network for NN pmerge and deep networks for NN pM , NN pY and NN pZ .
We also ﬁne-tune the network
NN qM for the approximate posterior q since M is involved in the inference of Z .
In sum, in ﬁnetuning the selective part of the deep CAMA model is trained to maximize the following objective: [Equation]
[Equation]
Notice that there may exist inﬁnitely many manipulations and it is impossible to observe all of them at training time.
Therefore by ﬁne-tuning at test-time, the model can be adapted to any unseen manipulation which is desirable in many real-life applications.
As shown in our experiments, the proposed deep CAMA model and the training methods are capable of improving the robustness of the generative classiﬁer to unseen manipulations.
3. D EEP CAMA FOR GENERIC MEASUREMENT DATA
We now discuss an even more general version of deep CAMA to handle multimodality in measurement data.
To predict the target variable Y in a directed acyclic graph, only variables in the Markov
[Figure] [Figure]
blanket of Y (shown in Figure 5) are needed.
This includes the parents ( A ), children ( X ), and co-parents ( C ) of the target Y .
Similar to the single modal case above, here a valid manipulation can only be independent mechanisms applied to X or C to ensure that Y does not change and the relationship from Y to X does not change.
We design the generic deep CAMA (shown in Figure 6) following the causal process in Figure 5.
Unlike discriminative DNNs where A , C and X are used together to predict Y directly, we consider the full causal process and treat them separately.
Building on the deep CAMA for single modality data, we add the extra consideration of the parent and observed co-parent of Y , while modelling the latent unobserved cause in Z and potential manipulations in M .
We do not need to model manipulation on C as they are out of the Markov Blanket of Y .
Thus, our model and the approximate inference network are deﬁned as:
[Equation]
Training, ﬁne-tuning and prediction proceed in the same way as in the single modality deep CAMA (Section 3.1) with do ( m ) operations and Monte Carlo approximations.
As we only ﬁne-tune the networks that are dependent on M , using similar reasoning one can show that the multimodality deep CAMA is robust to manipulations directly on the effect variable X .
Our proposed model is also robust to manipulations on the co-parents C by design.
By our deﬁnition of valid manipulation, perturbing C is valid as only causes the changes in X .
If the underlying causal relationship between C and X remains the same, and the trained model learns p ( x | y, c ) perfectly , then our model is perfectly robust to such changes.
This is because we use Bayes’ rule for prediction:
[Equation]
and the manipulations on C (thus changing X ) do not affect the conditional distribution p ( x | y, c ) in the generative classiﬁer (Eq. 11).
In contrast, discriminative DNNs concatenate X , C , A together and map these variables to Y , therefore they are sensitive to manipulations on C and/or X .
E XPERIMENTS
In this section, we ﬁrst show the robustness of our proposed deep CAMA for image classiﬁcation using both MNIST and a binary classiﬁcation task derived from CIFAR-10.
Then, we demonstrate the behaviour of our generic deep CAMA for measurement data.
We evaluated the perfromance of CAMA on both manipulations such as shifting and adverserial examples generated using the
CleverHans package (Papernot et al., 2018).
More results with different DNN architectures and different manipulations are shown in the appendix.
4. R OBUSTNESS TEST ON IMAGE CLASSIFICATION WITH D EEP CAMA
We ﬁrst demonstrate the robustness of our model against vertical (VT) and horizontal (HT) shifts.
Details such as network architectures are presented in the appendix.
The experiments are repeated for 5 times, and on MNIST, the results are stable and the variances are not visible in the plot. [Figure]
[Figure]
used
[Figure]
ﬁne
[Figure]
Training with clean MNIST data only.
Figure 7 shows the results for deep CAMA trained on clean data only.
Deep CAMA without ﬁne-tuning (orange lines) perform similarly to a DNN (blue lines) on horizontally shifted images
[Figure]
but it is more robust to vertical shifts.
The advantage of deep
CAMA is clear when ﬁne-tuning is used at test time (green lines): ﬁne-tuning on noisy test data with the same shift clearly improves the robustness of the network (panels 7(b) and 7(d)).
We further inspect the generalization of deep CAMA to unseen manipulation after ﬁne-tuning in panels 7(a) and 7(e).
The robustness results of ﬁne-tuned models are similar or even slightly better than the models without ﬁne-tuning.
This clearly shows that our model is capable of learning manipulations in an unsupervised manner, without deteriorating the generalization ability to unseen manipulations.
Lastly, panels 7(c) and 7(f) show the robustness of our model to both shifts when both types of manipulation are used for ﬁne-tuning, and we see clear improvements over both manipulations.
Training with augmented MNIST data We explore the setting where the training data is augmented with noisy data.
As discussed in Section 3.1, here deep CAMA naturally learns disentangled representation due to its independent mechanism design.
Indeed this is conﬁrmed by Figure 9, where panel 9(b) shows the reconstructions of noisy data from panel 9(a) with do ( m = 0) .
In this case the model keeps the identity of the digits but moves them to the center of the image.
Recall that do ( m = 0) corresponds to clean data which contains centered digits.
This shows that deep CAMA can disentangle the intrinsic unknown style Z and the shifting manipulation variable M .
We show the robustness results of deep CAMA with augmented training in Figure 10 (cf.
Figure 1).
Here shift range 0 .
5 is used to augment the training data.
Take the vertical shift test in panel 10(a) for example.
When vertically shifted data are augmented to the training set, the test performance without ﬁne-tuning (green line) is signiﬁcant better. Further, ﬁne-tuning (brown line) brings in even
[Figure] [Figure]
[Figure]
larger improvement for large scale shifts.
On the other hand, when using horizontally shifted data in training, deep CAMA’s robustness on vertically shifted data also improves (red line), which is different from discriminative DNNs overﬁtting behaviour (Figure 1).
Therefore deep CAMA shows signiﬁcant advantage over discriminative DNNs as its robustness to unseen manipulations can be improved by observing other related manipulations .
Our model does not overﬁt to a speciﬁc type of manipulations, at the same time further ﬁne-tuning can always improve the robustness against new manipulations in the test set (pink line).
The same conclusion holds in panel 10(b).
We also quantify the amount of noisy data required for ﬁne-tuning in order to improve the robustness of deep CAMA models.
As shown in Figure 8, even using 1% of the noisy data is sufﬁcient to learn the vertical shift manipulation presented in the test set.
Adversarial Attack Test on MNIST We further test deep CAMA’s robustness against two adversarial attacks: fast gradient sign method (FGSM)
(Goodfellow et al., 2014) and projected gradient descent (PGD) (Madry et al., 2017).
Note that, these attacks are specially developed for images with the small perturbation constraint.
However, theses attack does not have guarantee to be valid by our deﬁnition as the manipulation depends on the class label Y , which has the risk of changing the ground-truth label.
Such risk has also been discussed in Elsayed et al.
[Figure]
c [Figure] Figure 11 show the results comparing CAMA and the
DNN; both are trained on clean images only.
CAMA is signiﬁcantly more robust to both attacks than DNN (orange line), and with ﬁnetuning, CAMA shows additional 20% − 40% accuracy increase.
We also show the clean data test accuracy after ﬁne-tuning maintains to be the same thanks to our causal consistent model design.
Adversarial attack test on natural image classiﬁcation
The last experiment in this section evaluates the adversarial robustness of deep CAMA when trained on natural images.
In this case we follow Li et al.
and consider CIFAR-binary , a binary classiﬁcation dataset containing airplane and frog images from CIFAR-10. We choose to work with CIFAR-binary because VAE-based
[Figure] [Figure]
fully generative classiﬁers are less satisfactory for classifying clean CIFAR-10 images ( < 50% clean test accuracy).
The deep CAMA model trained with data augmentation (adding Gaussian noise with standard deviation 0.1, see objective (5)) achieves 88 .
85% clean test accuracy on CIFAR-binary, which is on par with the results reported in Li et al.
For reference, a discriminative CNN with 2 × more channels achieves 95 .
60% clean test accuracy.
Similar to previous sections we apply
FGSM and PGD attacks with different (cid:15) values to both deep CAMA and the discriminative CNN, and evaluate classiﬁcation accuracies on the adversarial examples before and after ﬁnetuning.
Results are reported in Figure 12.
For both FGSM and PGD tests, we see that deep CAMA, before ﬁnetuning, is signiﬁcantly more robust to adversarial attacks when compared with a discriminative
CNN model.
Regarding ﬁnetuning, although PGD with large distortion ( (cid:15) = 0 . 2 ) also fools the ﬁnetuning mechanism, in other cases ﬁnetuning still provides modest improvements ( 5% to 8% when compared with the vanilla deep CAMA model) without deteriorating test accuracy on clean data.
Combined with adversarial robustness results on MNIST, we conjecture that with a better generative model on natural images the robustness of deep CAMA can be further improved.
4. R OBUSTNESS TEST ON MEASUREMENT BASED DATA WITH GENERALIZED D EEP CAMA
Our causal view on valid manipulations allows us to test the robustness of models to generic measurement data.
Unfortunately, there exists no public dataset with multiple variables where ground truth causal relationships are known.
Therefore we generate synthetic data (see appendix) following a causal process, and test the performance of the generic deep CAMA on this measurement based data.
Here we use Gaussian variables for A , C and X , and categorical variables for Y .
All the ground truth causal relationships are nonlinear (quadratic mainly).
Manipulation Test First, we test manipulations on co-parents, C , while keeping the ground truth causal inﬂuence from C to X static.
Thus, both C and X change.
We manipulate C by shifting it up or down, which is a reasonable analogy to the noisiness in measurement data.
For example, in medical measurement data, different doctors may have different subjective standards while examining the patients, thus the same measurement can be shifted up or down.
Figure 14 shows the result: compared to a discriminatively trained DNN, deep CAMA is signiﬁcantly more robust to a wide range of manipulations.
However, when the range of the shifting manipulations increases, the classiﬁcation accuracy of the discriminative DNN drops drastically.
This conﬁrms our theory in
Section 3.2 that manipulations in C do not affect the decision making of deep CAMA, therefore our model is more robust to manipulation on co-parents as compared to discriminative DNNs.
Figure 15 shows the performance of the generic deep CAMA when the children X are manipulated, and the model only sees clean data at training time.
While deep CAMA achieves the same accuracy as a discriminative DNN on clean data, it is again signiﬁcantly more robust to manipulations even without ﬁne-tuning (the orange line vs the blue line).
With ﬁne-tuning (green line), the robustness of deep CAMA is further improved, especially when the amount of distortion is large.
The red line shows that deep CAMA’s test accuracy on clean data, which does not drop after ﬁne-tuning on different shifts.
This further conﬁrms that during test time, ﬁne-tuning learns the inﬂuence of M without affecting the causal relationships between Y and Z .
Adversarial Attack Test
Lastly we evaluate the adversarial robustness of the generalized CAMA model.
We only allow attacks on the children X and coparents C to be consistent with our deﬁnition of valid attacks.
This applies to both DNN and CAMA.
Figure 13 shows the results in terms of test accuracy with adversarial examples generated using FGSM and PGD attack methods.
Again deep CAMA demonstrate signiﬁcantly improved robustness against adversarial attacks, and ﬁne-tuning further provides improvements on robustness while keeping high accuracy on clean test examples.
R ELATED W ORK
Adversarial robustness Adversarial attacks can easily fool a discriminative DNN for vision/speech/language modelling tasks by adding imperceptible perturbations (Carlini & Wagner, 2018;
Alzantot et al., 2018; Carlini & Wagner, 2017b; Szegedy et al., 2013; Papernot et al., 2017).
Adversarial training (Madry et al., 2017; Tram`er et al., 2018) has shown some success in defending attacks, however, these techniques assume the knowledge of the adversary and present the perturbation to the model during training.
Still, a discriminative model after adversarial training is vulnerable to unseen manipulations.
Deep generative modelling has recently been applied as a defence mechanism to adversarial attacks.
Speciﬁcally, existing work considered de-noising adversarial examples before feeding these inputs to the discriminative classiﬁer (Song et al., 2018; Samangouei et al., 2018).
Very recently, research revisited (deep) generative classiﬁers and provided evidence that they are more robust to adversarial attacks (Li et al., 2018; Schott et al., 2019; Lee et al., 2018).
Causal learning Causal inference has a long history in statistical research (Spirtes et al., 2000;
Pearl, 2009; Peters et al., 2017; Pearl & Mackenzie, 2018).
Although it has fundamental importance, the causal view has not been widely incorporated to the robustness analysis of neural networks on unseen manipulations.
The most relevant work is in applying the existing causal views to transfer learning and domain adaption (Zhang et al., 2013; Stojanov et al., 2019;
Zhao et al., 2019;
Gong et al., 2016), where the difference in various domains are treated as either target shift or conditional shift from a causal perspective.
As an extension to the domain adaptation work, Rothenh ¨ ausler et al.
(2018); Heinze-Deml & Meinshausen (2017);
Arjovsky et al.
(2019) also discussed learning robust predictors across different domains.
However, in these approaches the domain is speciﬁed either explicitly or though exemplar paired points, thus an unseen manipulation is not explicitly considered.
By contrast, our proposed method does not rely on any given domain information.
Another related area is causal feature selection (Aliferis et al., 2010), where causal discovery is applied ﬁrst and features in the Markov Blanket of the prediction target are selected.
We also note that CAMA’s design is aligned with causal and anti-causal learning analyses (Sch ¨ olkopf et al., 2012; Kilbertus et al., 2018), in that CAMA models the causal mechanism Y → X and use Bayes’ rule for anti-causal prediction. Different from Sch¨olkopf et al. (2012), CAMA is not limited to only two endogenous variables; rather it provides more generic design handling latent causes that correspond to both intrinsic variations and data manipulations.
Disentangled representations Learning disentangled representations has become a hot topic of research in recent deep generative modelling literature.
A considerable amount of effort went to developing training objectives for variational auto-encoders, e.g. β -VAE (Higgins et al., 2017) and other information theoretic approaches (Kim & Mnih, 2018; Chen et al., 2018).
Additionally, different factorization structure in graphical model design has also been explored for disentanglement (Narayanaswamy et al., 2017; Li & Mandt, 2018).
D ISCUSSION
We have provided a causal view on the robustness of neural networks, showing that the vulnerability of discriminative DNNs is due to the lack of causal reasoning.
We deﬁned valid manipulations under this causal view, which are the manipulations on the children and/or the co-parents of the target variables, independent of the target and/or the cause of the target.
We further proposed a deep causal manipulation augmented model (deep CAMA), which follows the causal relationship in the model design, and can be adapted to unseen manipulations at test time.
Our model has demonstrated improved robustness, even without adversarial training.
When manipulated data are available, our model’s robustness increases for both seen and unseen manipulation.
Our framework is generic, however, manipulations can change over time, and a robust model should adapt to these perturbations in a continuous manner.
Our framework thus should be adapted to online learning or continual learning settings.
In future work, we will explore the continual learning setting of deep CAMA where new manipulations come in a sequence.
In addition, our method is designed for generic class-independent manipulations, therefore a natual extension would consider class-dependent manipulations where M is an effect of Y . Lastly out design excludes gradient-based adversarial attacks which is dependent on both the target and the victim model. As such attacks are commonly adopted in machine learning, we would also like to extend our model to such scenarios. A D ERIVATION D ETAILS A. T HE INTERVENTION ELBO
When training with clean data D = { ( x n , y n ) } , we set the manipulation variable M to a null value, e.g. do ( m = 0) . In this case we would like to maximise the log-likelihood of the intervened model, i.e. [Equation] This log-likelihood of the intervened model is deﬁned by integrating out the unobserved latent variable Z in the intervened joint distribution, and from do-calculus we have
[Equation] A variational lower-bound (or ELBO) of the log-likelihood uses a variational distribution q ( z |· ) [Equation] The lower-bound holds for arbitrary q ( z |· ) as long as it is absolutely continuous w.r.t.
the posterior distribution p θ ( z | x, y, do ( m = 0)) of the intervened model.
Now recall the design of the inference network/variational distribution in the main text:
[Equation]
where φ 1 and φ 2 are the inference network parameters of the corresponding variational distributions.
Performing an intervention do ( m = 0) on this q distribution gives
[Equation]
[Equation]
Deﬁning q
( z |· ) =
q φ 1 ( z | x, y, do ( m = 0)) and plugging-in it to eq.
(13) return the intervention
ELBO objective (3) presented in the main text.
A. T HE ELBO FOR UNLABELLED TEST DATA
The proposed ﬁne-tuning method in the main text require optimising the marginal log-likelihood log p θ ( x ) for x ∼ ˜ D , which is clearly intractable.
Instead of using a variational distribution for the unobserved class label Y , we consider the variational lower-bound of log p θ ( x, y ) for all possible y
=
y c :
[Equation]
[Equation]
Since both logarithm and exponent functions preserve monotonicity, and for all y c , c = 1 , ..., C we have log p θ ( x, y c ) ≥ ELBO ( x, y c ) , we have log p θ ( x, y c ) ≥ ELBO ( x, y c ) , ∀ c ⇒ p
θ ( x, y c ) ≥ exp[ ELBO ( x, y c )] , ∀ c ⇒ log p ( x ) = log (cid:34) C (cid:88) c
= p θ ( x, y c ) (cid:35) ≥ log (cid:34) C (cid:88) c = exp[ ELBO ( x, y c )] (cid:35)
[Equation]
which justiﬁes the ELBO objective (7) deﬁned in the main text.
B A DDITIONAL R ESULTS
CNN We also performed experiments using different DNN network architectures.
The convolution layers in CNN are designed to be robust to shifts.
Thus, we test these vertical and horizontal shifts with a standard CNN architecture as used in https://keras.io/examples/ cifar10_cnn/ .
4 convolution layers are used in this architecture.
Figure 16 shows the performance against different shifts.
We see that adding vertical shifts to the training data clearly harmed the robustness performances to unseen horizontal shifts as shown in 17(b).
Adding horizontal shifted images in training did not inﬂuences the performance on vertical shifts much.
Thus, we see that using different architectures of DNN, even the one that are designed to be robust to these manipulations, lack of generalization ability to unseen data is a common problem
[Figure]
[Figure]
Enlarge Network Size Here we exam whether network capacity has any inﬂuence on the robustness performance to unseen manipulation.
We use a wider network with [1024, 512, 512, 1024] units in each hidden layer instead of [512, 256, 126, 512] sized network in the paper.
Figureshows the robustness performance using this enlarged network.
We observe the similar degree of over-ﬁtting to the augmented data.
The penalization ability shows no improvement by enlarging the network sizes.
ZCA Whitening Manipulation Our result does not limited to shifts, it generalizes to other manipulations.
Figure 18 compare the result from training with clean images and training with ZCA whitening images added.
We see that adding ZCA whitening images in training harm both robustness against vertical shift and horizontal shift.
[Figure]
[Figure] [Figure]
Additional Figures
In addition to Figure 8, We also show the result testing with Vertical shift show in Figure 19, where a smaller N pM network ([dimM, 500, 500]) is used.
The conclusion is the same was using the vertical shift.
We need very few data for ﬁne-tune.
More than 1% data is sufﬁcient.
Similar as Figure 8, we show the result using different percentage of data for ﬁne-tuning in this experiment setting in 20.
C E XPERIMENTAL SETTINGS Network architecture
• MNIST experiments: – Discriminative DNN: The discriminate model used in the paper contains 4 densely connected hidden layer of [512 , 256 , 126 , 512] width for each layer.
ReLU activations and dropout are used with dropout rate [0 .
5] for each layer.
– Deep CAMA’s p networks: we use dim ( Y ) = 10 , dim ( Z ) = 64 and dim ( M ) = 32 .
NN pY : an MLP of layer sizes [ dim ( Y ) , 500 , 500] and ReLU activations.
NN pZ : an MLP of layer sizes [ dim ( Z ) , 500 , 500] and ReLU activations.
NN pM : an MLP of layer sizes [ dim ( M ) , 500 , 500 , 500 , 500] and ReLU activations.
NN p merge : an projection layer which projects the feature outputs from the previous networks to a 3D tensor of shape (4 , 4 , 64) , followed by 3 deconvolutional layers with stride 2, SAME padding, ﬁlter size (3 , 3 , 64 , 64) except for the last layer (3 , 3 , 64 , 1) .
All the layers use ReLU activations except for the last layer, which uses sigmoid activation.
– Deep CAMA’s q networks: NN qM : it starts from a convolutional neural network (CNN) with 3 blocks of { conv 3 × 3 , max-pool } layers with output channel size 64, stride 1 and SAME padding, then performs a reshape-to-vector operation and transforms this vector with an MLP of layer sizes [4 × 4 × 64 , 500 , dim ( M ) × 2] to generate the mean and log-variance of q ( m | x ) .
All the layers use ReLU activation except for the last layer, which uses linear activation.
NN qZ : ﬁrst it uses a CNN with similar architecture as NN Mq ’s CNN (except that the ﬁlter size is 5) to process x .
Then after the reshape-to-vector operation, the vector ﬁrst gets transformed by an MLP of size [4 × 4 × 64 , 500] , then it gets combined with y and m and passed through another MLP of size [500+ dim ( Y )
+ dim ( M ) , 500 , dim ( Z ) × 2] to obtain the mean and log-variance of q ( z | x, y, m ) .
All the layers use ReLU activation except for the last layer, which uses linear activation.
• Measurement data experiments: – Discriminative DNN: The A, C, X variables are concatenated to an input vector of total dimension 20 .
Then the DNN contains 3 densely connected hidden layer of [64 , 16 , 32] width for each layer, and output Y .
ReLU activations and dropout are used with dropout rate [0 .
5] for each layer.
– Deep CAMA’s p networks: we use dim ( Y ) = 5 , dim ( A ) = 5 , dim ( C ) = 5 , dim ( Z ) = 64 and dim ( M ) = 32 .
p ( y | a ) : an MLP of layer sizes [ dim ( A ) , 500 , 500 , dim ( Y )] , ReLU activations except for the last layer (softmax).
p ( x | y, c, z, m ) contains 5 networks: 4 networks { NN pY , NN pC , NN pZ , NN pM } to process each of the parents of X , followed by a merging network.
NN pY : an MLP of layer sizes [ dim ( Y ) , 500 , 500] and ReLU activations.
NN pC : an MLP of layer sizes [ dim ( C ) , 500 , 500] and ReLU activations.
NN pZ an MLP of layer sizes [ dim ( Z ) , 500 , 500] and ReLU activations.
NN pM : an MLP of layer sizes [ dim ( M ) , 500 , 500 , 500 , 500] and ReLU activations.
NN p merge : it ﬁrst start from a concatenation of the feature outputs from the above 4 networks, then transforms the concatenated vector with an MLP of layer sizes [500 × 4 , 500 , dim ( X )] to output the mean of x .
All the layers use ReLU activations except for the last layer, which uses linear activation.
– Deep CAMA’s q networks: q ( m | x ) : it uses an MLP of layer sizes [ dim ( X ) , 500 , 500 , dim ( M ) × 2] to obtain the mean and log-variance.
All the layers use ReLU activations except for the last layer, which uses linear activation.
q ( z | x, y, m, a, c ) : it ﬁrst concatenates x, y, m, a, c into a vecto, then uses an MLP of layer sizes [ dim ( X )+ dim ( Y )
+ dim ( M )+ dim ( A )+ dim ( C ) , 500 , 500 , dim ( Z ) × 2] to transform this vector into the mean and log-variance of q ( z | x, y, m, a, c ) .
All the layers use ReLU activations except for the last layer, which uses linear activation.
• CIFAR-binary experiments: – Discriminative CNN:
The discriminate model used in the paper is a CNN with 3 convolutional layers of ﬁlter width 3 and channel sizes [128, 128, 128], followed by a ﬂattening operation and a 2-hidden layer MLP of size [ 4 × 4 × 128 , 1000 , 1000 , 10 ].
It uses ReLU activations and max pooling for the convolutional layers.
– Deep CAMA’s p networks: we use dim ( Y ) = 10 , dim ( Z )
= 128 and dim ( M ) = 64 .
NN pY : an MLP of layer sizes [ dim ( Y ) , 1000 , 1000] and ReLU activations.
NN p Z : an MLP of layer sizes [ dim ( Z ) , 1000 , 1000] and ReLU activations.
NN p M : an MLP of layer sizes [ dim ( M ) , 1000 , 1000 , 1000] and ReLU activations.
NN p merge : an projection layer which projects the feature outputs from the previous networks to a 3D tensor of shape (4 , 4 , 64) , followed by 4 deconvolutional layers with stride 2, SAME padding, ﬁlter size (3 , 3 , 64 , 64) except for the last layer (3 , 3 , 64 , 3) .
All the layers use ReLU activations except for the last layer, which uses sigmoid activation.
– Deep CAMA’s q networks: NN qM : it starts from a convolutional neural network (CNN) with 3 blocks of { conv 3 × 3 , max-pool } layers with output channel size 64, stride 1 and SAME padding, then performs a reshape-to-vector operation and transforms this vector with an MLP of layer sizes [4 × 4 × 64 , 1000 , 1000 , dim ( M ) × 2] to generate the mean and log-variance of q ( m | x ) .
All the layers use ReLU activation except for the last layer, which uses linear activation.
NN qZ : ﬁrst it re-uses NN qM CNN network for feature extraction on x .
Then after the reshape-to-vector operation, the vector gets combined with y and m and passed through another MLP of size [4 × 4 × 64+ dim ( Y )+ dim ( M ) , 1000 , 1000 , dim ( Z ) × 2] to obtain the mean and log-variance of q ( z | x, y, m ) .
All the layers use ReLU activation except for the last layer, which uses linear activation.
Measurement data generation We set the target Y to be categorical, its children, co-parents and parents are continuous variables.
The set 5 classes for Y , and Y has 10 children variables andco-parents variables, also one 5 dimensional parents.
Parents ( A ) and co-parents ( C ) are generated by sampling from a normal distribution.
We generate
Y using structured equation Y = f
y ( A ) + σ Y .
We use f y =
argmax g
( A ) and g () is a quadratic function 0 .
2 ∗ A 2 − 0 .
8 A .
σ Y is the Gaussain noise.
To generate the children X = f ( Y, C ) + σ x , we also used quadratic function f
and the parameters were sampled from a Gaussian distribution.
As in the experiment, we were using ﬁxed scale shift, we also added a normalize the children before adding the Gaussian random noise σ x .
So that all observations are in similar scale.
Other For MNIST experiments, we uses 5% of the training data as the validation set.
We used the training results with the highest validation accuracy for testing.
If not otherwise speciﬁed, 50% of noisy test data are used for ﬁne-tuning in the shift experiments and all data are used for ﬁne-tuning in the attack experiments.
For the experiments with measurement data.
We generated 1000 data in total.
We split, 500 data for testing, 450 for training and 50 for validation.
We used the training results with the highest validation accuracy for testing for both deep CAMA and for DNN.