A Causal View on Robustness  of Neural Networks
Authors: Cheng Zhang
Yingzhen Li
Venue:  (2020)
Keywords: Neural Network Robustness
Variational autoencoder (VAE)
Causality
Deep generative model

Abstract: We present a causal view on the robustness of neural networks against input manipulations, which applies not only to traditional classification tasks but also to general measurement data. Based on this view, we design a deep causal manipulation augmented model (deep CAMA) which explicitly models the manipulations of data as a cause to the observed effect variables. We further develop data augmentation and test-time fine-tuning methods to improve deep CAMA's robustness. When compared with discriminative deep neural networks, our proposed model shows superior robustness against unseen manipulations. As a by-product, our model achieves disentangled representation which separates the representation of manipulations from those of other latent causes.

Under review as a conference paper at ICLR 2020
A CAUSAL VIEW ON ROBUSTNESS OF NEURAL NETWORKS
Anonymous authors Paper under double-blind review
ABSTRACT
We present causal view the robustness of neural networks against input nipulations, which applies not only to traditional classification tasks but also to general measurement data. Based on this view, we design deep causal manipu- lation augmented model (deep CAMA) which explicitly models the manipulations of data as cause to the observed effect variables. We further develop data aug- mentation and test-time fine-tuning methods to improve deep CAMA's robustness. When compared with discriminative deep neural networks, our proposed model shows superior robustness against unseen manipulations. As by-product, our model achieves disentangled representation which separates the representation manipulations from those of other latent causes.
INTRODUCTION
Deep neural networks (DNNs) have great success in many real-life applications, however, they are easily fooled even by tiny amount of perturbation (Szegedy et al., 2013; Goodfellow et al., 2015; Carlini & Wagner, 2017b; Athalye et al., 2018). Lack of robustness hinders the application of DNNs to critical decision making tasks such uses in health care. To address this, deep learning prac- titioner may suggest training DNNs with datasets that are not only big but also diverse. Indeed, data augmentation and adversarial training have shown improvements in both the generalization and robustness of DNNs (Kurakin et al., 2016; Perez & Wang. 2017; Madry et al., 2017). Unfortunately, this does address the vulnerability of DNNs for unseen manipulations. For example, as shown in Figure 1, DNN trained on clean MNIST digits fails to classify shifted digits. Although observing (adversarial) perturbations of clean data in training improves robustness against that particular ma- nipulation (the green line), the DNN is still fragile when unseen manipulations line). Since is unrealistic to augment the training data towards all possible manipulations that many occur, principled method that fundamentally improves the robustness is much needed.
present (orange
On the other hand, humans naturally understand the independent causal mechanisms for visual recognition tasks, where the generative process of the perceived view is composed of modules that do not influence each other (Parascandolo et al., 2017). After learning the concept of an "elephant", child can identify the elephant in photo taken under any lightning condition, location, etc. Im- portantly, the elephant, the lightning condition, and the location are causes of the presented view in the photo. Therefore we argue that the incapability for causal reasoning (Pearl Mackenzie, 2018; Gopnik et al., 2004) is the reason of DNN's 's vulnerability to (adversarial) data manipulations.
This work discusses the robustness of DNNs from causal perspective. Our contributions are:
A causal view on robustness of neural networks. We argue from causal perspective that adversarial examples for a model can be generated by manipulations on the effect variables and/or their unseen causes. Therefore DNN's vulnerability to adversarial attacks is due to the lack of causal understanding.
A causal inspired deep generative model. We design causal deep generative model which takes into account the unseen manipulations of the effect variables. Accompanied with this model is a test-time inference method to learn unseen manipulations and thus improve classification accuracy on noisy inputs. Data augmentation techniques can also be safely applied to our model during training without deteriorating its generalization ability to un- seen manipulations. Compared to DNNs, experiments on both MNIST and a measurement- based dataset show that model is significantly more robustness to unseen manipulations.

---

Under review as a conference paper at ICLR 2020
1.0
1.0
0.9
0.9
0.8
0.8
0.7
00.7
Train
0.6
Train Hor Test Ver
0.6
Train Ver Test Hor
0.5
Train Ver test
0.5
Train Hor
0.4
0.4
0.3
0.3
0.2
0.2
0.
0.1
0.0
0.1 0.2 0.3
0.6 0.7 0.8 0.9
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.3 0.9
range
Horizontal shift range
(a) Test Vertical shift
(b) Test Horizontal shift
Figure I: Robustness results for DNNs against different manipulations on MNIST. Panels (a) and (b) show the accuracy classifying noisy test data generated by shifting the digits vertically (Ver) and horizontally (Hor). It shows that data augmentation during training makes generalization to unseen shifts worse (orange versus blue lines).
A CAUSAL VIEW ON ROBUSTNESS OF NEURAL NETWORKS
Discriminative DNNs are not robust to manipulations such as adversarial noise injection (Goodfel- low et al., 2015; Carlini & Wagner, 2017a; Athalye et al., 2018), rotation and shift. They do not understand the causal mechanisms of the data generating process, which leads to overfiting to nui- sance factors that are less related to the ground truth classification results. By exploiting the overfit to the nuisance factors, an adversary can easily manipulate the inputs to fool discriminative DNNs into predicting the wrong outcomes.
On the contrary, we as human can easily recognize an object in a scene and be indifferent to the changes in other aspects such as background, viewing angle, the presence of sticker the ob- ject, etc. More importantly, our recognition is not affected even when some of the perturbations, e.g. changes in the lighting condition, are significant. We argue that the main difference here is due to our ability to perform causal reasoming, which identifies independent mechanisms that are not causally related to the object recognition results (Freeman, 1994; Peters et al., 2017; Parascandolo et al., 2017). This leads to robust human perception to not only certain type of perturbations, but also many types of manipulations. Thus we argue that one should incorporate causal mechanisms into model design, and make the model robust on the level of different types of perturbations.
Before presenting our causally informed model, we first define a valid manipulation inputs in causal sense. A valid manipulation is perturbation on data, which only changes the effects, not the cause of the target. We visualize a causal graph in Figure 2, where the arrows indicate the cause- effect relationship between variables. Take hand-written digit classification for example, X is the image of a digit and Y is the class label. The appearance X is an effect of the digit number Y, latent causes Z such writing styles, and possible manipulations M, such rotation or translation. Changes to Z and M cause the appearance of X to change, but X still carries the same information about Y regardless of these perturbations, since Z, M and Y are independent mechanisms. Thus, any manipulation that does not influence the Y + relationship are valid manipulations. Humans are extremely robust to these manipulations while machine learning algorithms are vulnerable.
In summary, from the causal perspective, any manipulation M on data X, that is co-parent of Y, is valid manipulation. This definition includes many manipulations used in existing work on the robustness of neural networks, such as noise injection, shift and rotation (Engstrom et al., 2019). Ideally, machine learning model should be able generalize any valid manipulation, the same time training with manipulated data of certain types should never harm the model's robustness to unseen manipulations. However, discriminative deep learning models ignore the causal structure and consider X + Y only, which explains their vulnerability to data manipulations. Inspired by causal reasoning of humans, we propose a deep learning framework concerning the causal relationship.
3 THE CAUSAL MANIPULATION AUGMENTED MODEL.
We propose deep CAusal Manipulation Augmented model (deep CAMA), which takes into count the causal relationship for model design. Our proposed model is more robust to unseen manipulations effect variables, and more importantly, model can learn these manipulations
2

---

Under review as a conference paper at ICLR 2020
NN?
NNY
NNW
XNH
by
hz
Z
M
NN
NN
X
decoder teiwork
enccoder wetworkk
Figure 2: A simple ex- ample, where X is the effect of Y, Z and M.
Figure 3: Graphical presenta- tion of proposed causally con- sistent deep generative model for single modal data.
Figure 4: The network architecture. Shaded areas show the selective part for do(m) training and the fine-tune method, respectively.
without supervision. The robustness can be further improved by training-tine time data augmentation, without sacrificing the generalization ability to unseen manipulations. Below we first present the deep CAMA for single modality data, which focuses on predicting Y using X, and then present a generic deep CAMA for multimodality measurement data.
3.1 DEEP CAMA FOR SINGLE MODALITY DATA
The task of predicting Y from X covers a wide range of applications such as image/speech recogni- tion and sentiment analysis. Normally discriminative DNN takes X as input and directly predicts (the distribution of) the target variable Y. Generative classifiers, on the other hand, build generative model + X, and use Bayes' rule for predicting Y given X: p(yla)) = p(y)p(r /)/((z).
We design deep CAMA (Figure 3) following the causal relationship shown in Figure 2. Taking MNIST for example: Y is the label and X is the image, Z models the latent style of the digits, and M handles the manipulations that desire the model to be robust to. The model is defined as:
po(x, y, 2, m. =
m)
(1)
For efficient inference we follow the amortized inference approach in variational auto-encoders (Kingma & Welling, 2013; Rezende et al., 2014; Zhang et al., 2018) and define an inference network as the approximate posterior distribution:
(=, m y) = 4s, (z|ar, y. m.m.) m))0. (m|æ))
(2)
We use @ to denote all the parameters the encoder network and ş {Ø1, d2), where Φ1 is the parameter for the encoder network for the variational distribution 9ø, (z|ar, y, m). and de is used for the Go, (m[z) part. Note that we assume the dependence of M on X only in Z0g (m|a), which, as shall show later, allows deep CAMA learn unseen manipulations with unlabelled noisy data.
The network architecture is presented in Figure 4. For the model, the cause variables Y, and M are first transformed into feature vectors hy, hz and hM. Later, these features are merged gether and then passed through another neural network to produce the distributional parameters of Po (rrl,, 2, m,). For the approximate posterior q. different networks used compute the distributional parameters of gez (mlz) and qo. (2 y. m), respectively.
Model training Assume that during training. the model observes clean data D {(rn yn)) only. In this case set the manipulation variable M to null value, e.g. do(m. = 0), and train deep CAMA by maximizing the likelihood function log p(x, y (m = 0)) under training data. Since this marginal distribution is intractable, instead maximize the intervention evidence lower-bound (ELBO) with do(m = 0), i.c. max9, Ep ELBO (x, y, do(m = 0))], with the ELBO defined as
(o y. do(m m = 0)) ELBO(z, y, do(m = 0)) := (2l. log
9s |z. do(m = 0))
(3)
log Po (xly, z. m 0)p(z)r(z)
-K (=la.y.==))
4o. (2 æ. m = 0)
3

---

Under review as a conference paper at ICLR 2020
See appendix A for detailed derivation. If noisy data D' is available during training, then similar to data augmentation and adversarial training (Goodfellow et al., 2015; Tramèr et al., 2018; Madry et al., 2017), we can augment the training data with this noisy data. We still use the intervention ELBO (3) for clean data. For the manipulated instances, we can either use the intervention ELBO with do(m= mo) when the noisy data D {(mo(x)))) y)} } is generated by known manipulation mo, or, as done in our experiments, infer the latent variable M for unknown manipulations. This is achieved by maximizing the ELBO on the joint distribution log y) using noisy data:
ELBO v) == (s.(mm)) log (o U3 2,) q (2, m m(r,))
(4)
and therefore the total loss function to be maximized defined as
uugg (,, ②) = X (ELBO y, do(m = 0))l + (1 - X)E [EBBO(2, v)].
(5)
Our causally consistent model effectively disentangles the latent representation: Z models the un- known causes in the clean data, such as personal writing style; and M models possible manipulations which the model should be robust to, such as shift, rotation, noise etc. Due independent mech- anism assumptions in causality, the influence of Y. Z and M on X can be independently applied. Thus, with our model design, we can also ensure that the dependencies Y + X and Z + X are not affected by noisy data present during training. As a result, deep CAMA's can still generalize to un- seen manipulations even after seeing lots of noisy datapoints from other manipulations, in contrast to the behavior of discriminative DNNs shown in Figure 1.
In general the test data D can be noisy, and we would like our model to be robust to the unseen manipulated test data, Thus, at test-time, M is unknown, and deep CAMA classifies an unseen test data 2", using Monte Carlo approximation to Bayes' rule with samples m gog (m/x), 26 20,,(21 (=|a", "1la",,u ,e,, m"):
Prediction
"oly") = softmax p(x*)
K
Po (alyy, = m p()o()))p()) qs, feflx,,0 yer Me)m")
p(y"
log
(6)
cml
In addition, deep CAMA can be adapted the unseen manipulations present at test time without labels on the noisy data. From the causal graph, the conditional distributions p( ((J{) and p(X) ( are invariant to the interventions on X based on the independent mechanism assumption (Peters et al.. 2017), however, we would like to learn the manipulation mechanism M + X. As shown in Figure 4, for the generative model, we only fine-tune the networks that are dependent only on M, i.e. NN ay by maximizing the ELBO of the marginal distribution log p(r):
ELBO(x) :- log Σ exp exp/OLLOO v.)]
To reduce the possibly negative effect of fine-tuning to model generalization, we use shallow network for NN and deep networks for NN ar. NNP- and NN We also fine-tune the network NN;, for the approximate posterior q since M is involved in the inference of Z. In sum, in fine- tuning the selective part of the deep CAMA model is trained to maximize the following objective:"
La (0, Φ) = aE [ELOO v)] + (1 - (ELBO((())
Notice that there may exist infinitely many manipulations and it is impossible to observe all of them training time. Therefore by fine-tuning at test-time, the model can be adapted to any unseen manipulation which is desirable in many real-life applications. As shown in our experiments, the proposed deep CAMA model and the training methods are capable of improving the robustness of the generative classifier to unseen manipulations.
3.2 DEEP CAMA FOR GENERIC MEASUREMENT DATA
We now discuss an even more general version of deep CAMA to handle multimodality in measure- ment data. To predict the target variable Y in directed acyelic graph, only variables in the Markov
One can also the intervention ELBO for the clean training data.
4

---

Under review as a conference paper at ICLR 2020
A
A
M
C
C2
X1
Figure 5: The Markov Blanket of target vari- able Y
Figure 6: Graphical presentation of proposed causal deep generative model for generic measurement modal data.
blanket of Y (shown in Figure 5) needed. This includes the parents (A), children (X), and co-parents (C) of the target Y. Similar to the single modal case above, here a valid manipulation can only be independent mechanisms applied X or C to ensure that Y does not change and the relationship from Y to X does not change.
We design the generic deep CAMA (shown in Figure 6) following the causal process in Figure 5. Unlike discriminative DNNs where A, C' and X are used together to predict Y directly, we consider the full causal process and treat them separately. Building on the deep CAMA for single modality data, we add the extra consideration of the parent and observed co- to-parent of Y, while modelling the latent unobserved cause Z and potential manipulations M. We do not need to model manipulation on C' as they are out of the Markov Blanket of Y. Thus, our model and the approximate inference network are defined as:
p0(x, (I, y, 2, m, a, c) p(a)p(m)p(()p((ppp (yla)pe, (x)8, (xly,,,,,m)
(9)
g0(z, m|
c) = 4o, (z|æ, y. c) (m|r).
(10)
Training, fine-tuming and prediction proceed in the same way as in the single modality deep CAMA (Section 3.1) with do(m) operations and Monte Carlo approximations. As we only fine-tune the networks that are dependent M, using similar reasoning deep CAMA is robust to manipulations directly on the effect variable X.
show that the multimodality
Our proposed model is also robust to manipulations of valid manipulation, perturbing C' valid as only causes the changes in X. If the underlying causal relationship between and X remains the same, and the trained model learns p(æly, e) perfectly, then our model is perfectly robust to such changes. This is because we use Bayes' rule for prediction:
the co-parents C by design. By our definition
p(yla, x, = plulayptalppchppttu.cc = p(ula)p(zlv.c)
(11)
p(a)p(c) (( S,, (,p(vla)(()yy.c))
and the manipulations on C' (thus changing X) do not affect the conditional distribution p(Zly, c) in
the generative classifier (Eq. 11). contrast, discriminative DNNs concatenate X, C, together and map these variables to Y. therefore they are sensitive to manipulations on C' and/or X.
EXPERIMENTS
In this section, we first show the robustness of our proposed deep CAMA for image classification using both MNIST and binary classification task derived from CIFAR-IO. Then, we demonstrate the behaviour of our generic deep CAMA for measurement data. We evaluated the perfromance of CAMA on both manipulations such shifting and adverserial examples generated using the CleverHans package (Papernot et al., 2018). More results with different DNN architectures and different manipulations are shown in the appendix.
4.1 ROBUSTNESS TEST ON IMAGE CLASSIFICATION WITH DEEP CAMA
We first demonstrate the robustness of our model against vertical (VT) and horizontal (HT) shifts. Details such network architectures presented in the appendix. The experiments repeated for 5 times, and on MNIST, the results are stable gnd the variances are not visible in the plot.

---

Under review as a conference paper at ICLR 2020
0.8
Durs
Outs mlo FT
0.7
Our w FT Ver
Ous m FT Hot
0.77
Ours m FT Both
0.6
0.6
0.5
0.4
0.4
0.4
0.3
0.3
0.3
0.2
0.2
0.2
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
0.1
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7
Horizontal shift range
Horizontal shift range
Horizontal shift range
(a) Finetune VT test HT
(b) Finetune HT test HT
(c)
Finetune Both test HT
Accuracy
0.8
0.7
Our FT Ver
wo
Out m FT Hor
Accurac
677
Ours m FT Both
0.6
0.5
0.4
0.3
0.3
0.3
0.2
0.2
0.2
0.0 0.1 0.2 0.3 0.4 0.5 066 0.7 0.8 0.9
0.0 0.1 0.2 0.3 0.5 0.6 0.7 0.8 0.9
0.1
0.0 0.1 0.2 0.3 0.4 0,5 0.6 0.7
Vertical shift range
Vertical shift range
Vertical shift range
(d) Finetune VT test VT
(c) Finetune HT test VT
(f) Finetune Both test VT
Figure 7: The first shows the results of testing the model robustness against horizontal shifts and the second row shows the results against vertical shifts.
7
4
0.9
1.0
DNN
0.55
0
3
0
9
5
9
7
3
4
Accuracy
0.8
7
6
27
7
6
4
O
4
O
I
0.7
0.01
0.75
2
3
3
7 2
7
2
0.6
0.1
2
4
3 5
4 9 3
4 S
0.4
3
5
6
6
5
4
0.3
)
7
4
4
&
9
3
7
4 3
6
O 7
0.b.
0.1 0.2
0.3 0.4 D.5 0.6 0.7 0.8 0.9
7
7
Vertical shift range
R
4
3
6
6
4
7
3
6
4
G
9
Figure 8: Performance regarding different percentages of test data used for fine-tuning manipulation
(a) Vertically shifted training (b) do(m=)) with the 2 and y data
from the vertical shifted data
Figure 9: Visualization of the disentangled representation.
Training with clean MNIST data only. Figure 7 shows the results for deep CAMA trained on clean data only. Deep CAMA without fine-tuning (orange lines) perform similarly to DNN (blue lines) on horizontally shifted images, but it is more robust vertical shifts. The advantage of deep CAMA is clear when fine-tuning is used at test time (green lines): fine-tuning on noisy test data with the same shift clearly improves the robustness of the network (panels 7(b) and 7(d)). We further inspect the generalization of deep CAMA to unseen manipulation after fine-tuning in panels 7(a) and 7(e). The robustness results of fine-tuned models are similar or even slightly better than the models without fine-tuning. This clearly shows that our model is capable of learning manipulations an unsupervised manner, without deteriorating the generalization ability to unseen manipulations. Lastly, panels 7(c) and 7(f) show the robustness of our model to both shifts when both types of manipulation are used for fine-tuning, and we see clear improvements over both manipulations.
Training with augmented MNIST data We explore the setting where the training data is aug- mented with noisy data. As discussed Section 3.1, here deep CAMA naturally learns disentangled representation due to its independent mechanism design. Indeed this is confirmed by Figure 9, where panel 9(b) shows the reconstructions of noisy data from panel 9(a) with do(m = 0). In this case the model keeps the identity of the digits but moves them to the center of the image. Recall that do(m = 0) corresponds to clean data which contains centered digits. This shows that deep CAMA disentangle the intrinsic unknown style Z and the shifting manipulation variable M.
We show the robustness results of deep CAMA with augmented training in Figure 10 (cf. Figure 1). Here shift range 0.5 is used to augment the training data. Take the vertical shift test in panel 10(a) for example. When vertically shifted data are augmented to the training set, the test performance without fine-tuning (green line) is significant better. Further, fine-tuning (brown line) brings in even

---

Under review as a conference paper at ICLR 2020
0.9
0.9
0.8
Ours we FT train clean
Accurad
0.7
5 0.7
train Var
0.6
0.6
Ours we FT train Her
0.5
FT dean
0.4
0.4
Ours FT train Ver
Ours train Hor
0.2
0.2
0.0 0.1 0.2 0.3 0.4 0,5 0.6 0.7 0.8 0.9
0.1
0.2
0.3
0.4 0.5 0.6 0.7 0.9
Vertical shift range
Horizontal shift range
(a) Test Vertical shift
(b) Test horizontal shift
Figure 10: Performance of our model against different manipulation (c.f. Figure 1).
0.8
0.8
9.6
0.6
06
0.5
0.4
Ds
B4
Gb
CANA REFWOO
14R% cEP AUU
GOND NIS
GIAE
CA
1.
CMN
0.
R4
0.22
0.55
02
101
0.8 1.55
22
0.1
02
04
05
01
0.2
03
05
ag
FESH eos
PGD ops
(a) FGSM
(b) PGD
(a) FGSM
(b) PGD
Figure 12: Test accuracy on adversarial examples crafted on CIFAR-binary data.
Figure 13: Test accuracy adversarial examples crafted on measurement data.
larger improvement for large scale shifts. On the other hand, when using horizontally shifted data training, deep CAMA's robustness on vertically shifted data also improves (red line), which is different from discriminative DNNs overfitting behaviour (Figure 1). Therefore deep CAMA shows significant advantage over discriminative DNNs as its robustness to unseen manipulations can be improved by observing other related manipulations. Our model does not overfit to specific type of manipulations, at the same time further fine-tuning can always improve the robustness against new manipulations in the test set (pink line). The same conclusion holds in panel 10(b).
We also quantify the amount of noisy data required for fine-tuning in order to improve the robustness of deep CAMA models. As shown in Figure even using 1% of the noisy data is sufficient to learn the vertical shift manipulation presented in the test set.
Adversarial Attack Test on MNIST We further test deep CAMA's robustness against two ad- versarial attacks: fast gradient sign method (FGSM) (Goodfellow et al., 2014) and projected gra- dient descent (PGD) (Madry et al., 2017). Note that, these attacks are specially developed for im- ages with the small perturbation constraint. However, theses attack does have guarantee to be valid by our definition as the manipulation depends on the class label Y, which has the risk of changing the ground-truth label. Such risk has also been discussed in Elsayed al. (2018). Figure 11 show the results comparing CAMA and the DNN; both are trained on clean images only. CAMA is significantly more robust to both attacks than DNN (orange line), and with fine- tuning, CAMA shows addi- tional 20% 40% accuracy increase. We also show the clean data test accuracy after fine-tuning maintains to be the same thanks to our causal consistent model design.
0.6
Accuracy
D6 CANA wio FT CANA FT CAaA
0.4
0.2
0.0
0.1
0.2
03
0.4
0.5
FGSM
ops
PGD
Figure 11: Test accuracy on MNIST adversarial examples.
Adversarial attack test on natural image classification The last experiment in this section eval- uates the adversarial robustness of deep CAMA when trained natural images. In this case follow Li et al. (2018) and consider CIFAR-binary, a binary classification dataset containing air- plane and frog images from C10. We choose work with CIFAR-binal because VAE-based

---

Under review as a conference paper at ICLR 2020
1.0
09
20000
2-0
0.00
2.
0um wi
CAR
CRN
0am FT
05
Ous w
0.00
0.0
155
24.
3.5
1.5
2.5
%
3.8
Shift
(a) Shift Up
(b) Shift Down
(a) Shift Up
(b) Shift Down
Figure 14: Manipulate co-parents
Figure 15: Manipulate children
fully generative classifiers are less satisfactory for classifying clean CIFAR- 10 images (< 50% clean test accuracy). The deep CAMA model trained with data augmentation (adding Gaussian noise with standard deviation 0.1, objective (5)) achieves 88.85% clean test accuracy CIFAR-binay which is on par with the results reported in Li et al. (2018). For reference, a discriminative CNN with 2x more channels achieves 95.60% clean test accuracy. Similar previous sections we apply FGSM and PGD attacks with different values to both deep CAMA and the discriminative CNN, and evaluate classification accuracies on the adversarial examples before and after finetuming.
Results are reported in Figure 12. For both FGSM and PGD tests, we see that deep CAMA, before finetuning, is significantly more robust to adversarial attacks when compared with discriminative CNN model. Regarding finetuning, although PGD with large distortion (c 0.2) also fools the finetuning mechanism, in other cases finetuning still provides modest improvements (5% 8% when compared with the vanilla deep CAMA model) without deteriorating test accuracy on clean data. Combined with adversarial robustness results on MNIST, we conjecture that with a better generative model on natural images the robustness of deep CAMA can be further improved.
4.2 ROBUSTNESS TEST ON MEASUREMENT BASED DATA WITH GENERALIZED DEEP CAMA
Our causal view on valid manipulations allows us to test the robustness of models to generic mea- surement data. Unfortunately, there exists no public dataset with multiple variables where ground truth causal relationships are known. Therefore we generate synthetic data (see appendix) following causal process, and test the performance of the generic deep CAMA this measurement based data. Here we use Gaussian variables for A, C' and X, and categorical variables for Y. All the ground truth causal relationships are nonlinear (quadratic mainly).
Manipulation Test First, we test manipulations on co-parents, C, while keeping the ground truth causal influence from C to X static. Thus, both C' and X change. We manipulate C' by shifting it up or down, which is a reasonable analogy to the noisiness in measurement data. For example, in medical measurement data, different doctors may have different subjective standards while ex- amining the patients, thus the same measurement can be shifted up or down. Figure 14 shows the result: compared to diseriminatively trained DNN, deep CAMA is significantly more robust to a wide range of manipulations. However, when the range of the shifting manipulations increases, the classification accuracy of the discriminative DNN drops drastically. This confirms our theory in Section 3.2 that manipulations in C' do not affect the decision making of deep CAMA, therefore our model is more robust to manipulation on co-parents as compared to discriminative DNNs.
Figure 15 shows the performance of the generic deep CAMA when the children X are manipulated, and the model only sees clean data at training time. While deep CAMA achieves the same accuracy as a discriminative DNN on clean data, it is again significantly more robust to manipulations even without fine-tuning (the orange line vs the blue line). With fine-tuning (green line), the robustness of deep CAMA is further improved, especially when the amount of distortion is large. The red line shows that deep CAMA's test accuracy on clean data, which does not drop after fine-tuning on different shifts. This further confirms that during test time, fine-tuning learns the influence of M without affecting the causal relationships between Y and Z.
Adversarial Attack Test Lastly we evaluate the adversarial robustness of the generalized CAMA model. We only allow attacks on the children X and coparents C' to be consistent with our definition of valid attacks. This applies to both DNN and CAMA. Figure 13 shows the results in terms of test accuracy with adversarial examples generated using FGSM and PGD attack methods. Again deep CAMA demonstrate significantly improved robustness against adversarial attacks, and fine-tuning further provides improvements robustness while keeping high accuracy clean test examples.
8

---

Under review as a conference paper at ICLR 2020
5 RELATED WORK
Adversarial robustness Adversarial attacks can easily fool discriminative DNN for vi- sion/speech//anguagee modelling tasks by adding imperceptible perturbations (Carlini & Wagner, 2018: Alzantot et al., 2018: Carlini & Wagner, 2017b; Szegedy al., 2013; Papernot et al., 2017). Adversarial training (Madry et al., 2017; Tramèr et al., 2018) has shown some success in defending attacks, however, these techniques assume the knowledge of the adversary and present the perturba- tion to the model during training. Still, a discriminative model after adversarial training is vulnerable to unseen manipulations. Deep generative modelling has recently been applied as a defence mecha- nism to adversarial attacks. Specifically, existing work considered de-noising adversarial examples before feeding these inputs to the discriminative classifier (Song et al., 2018; Samangouei et al., 2018). Very recently, research revisited (deep) generative classifiers and provided evidence that they are more robust to adversarial attacks (Li et al., 2018; Scholt et al., 2019; Lee et al., 2018).
Causal learning Causal inference has long history in statistical research (Spirtes et al., 2000; Pearl, 2009; Peters et al., 2017; Pearl & Mackenzie, 2018). Although it has fundamental importance, the causal view has not been widely incorporated to the robustness analysis of neural networks on unseen manipulations. The most relevant work is in applying the existing causal views to transfer learning and domain adaption (Zhang et al., 2013; Stojanov et al., 2019; Zhao et al., 2019; Gong et al., 2016), where the difference in various domains are treated as either target shift or conditional shift from a causal perspective. As an extension to the domain adaptation work, Rothenhäusler et al. (2018); Heinze-Deml & Meinshausen (2017); Arjovsky et al. (2019) also discussed learning robust predictors across different domains. However, in these approaches the domain is specified either explicitly though exemplar paired points, thus unseen manipulation is not explicitly considered. By contrast, our proposed method does not rely on any given domain information. Another related area is causal feature selection (Aliferis al., 2010), where causal discovery is applied first and features in the Markov Blanket of the prediction target are selected. We also note that CAMA's design is aligned with causal and anti-causal learning analyses (Schölkopf et al., 2012: Kilbertus et al., 2018), in that CAMA models the causal mechanism Y + X and use Bayes' rule for anti-causal prediction. Different from Schölkopf al. (2012), CAMA is not limited to only two endogenous variables; rather it provides more generic design handling latent causes that correspond to both intrinsic variations and data manipulations.
Disentangled representations Learning disentangled representations has become hot topic of research in recent deep generative modelling literature. A considerable amount of effort went to developing training objectives for variational auto-encoders, e.g. B-VAE (Higgins et al., 2017) and other information theoretic approaches (Kim & Mnih, 2018; Chen et al., 2018). Additionally, dif- ferent factorization structure in graphical model design has also been explored for disentanglement (Narayanaswamy et al.. 2017; Li & Mandt, 2018).
6 DISCUSSION
We have provided a causal view the robustness of neural networks, showing that the vulnerability of discriminative DNNs is due to the lack of causal reasoning. We defined valid manipulations under this causal view, which are the manipulations on the children and/or the co-parents of the target variables, independent of the target and/or the cause of the target. We further proposed a deep causal manipulation augmented model (deep CAMA), which follows the causal relationship in the model design, and can be adapted to unseen manipulations at test time. Our model has demonstrated improved robustness, even without adversarial training. When manipulated data are available, our model's 's robustness increases for both seen and unseen manipulation.
Our framework is generic, however, manipulations can change over time, and a robust model should adapt to these perturbations in a continuous manner. Our framework thus should be adapted to online learning or continual learning settings. In future work, we will explore the continual learning setting of deep CAMA where new manipulations come in a sequence. In addition, our method is designed for generic class- ftass-independen manipulations, therefore a natual extension would consider olaes-dependent manipulations where M is effect Y. Lastly out design excludes gradient-based adversarial attacks which is dependent on both the target and the victim model. As such attacks are commonly adopted in machine learning, we would also like extend our model to such scenarios.

---

Under review as a conference paper at ICLR 2020
REFERENCES
Constantin F Aliferis, Alexander Statnikov, Ioannis Tsamardinos, Subramani Mani, and Xenofon D Koutsoukos. Local causal and markov blanket induction for causal discovery and feature selec- tion for classification part i: Algorithms and empirical evaluation. Journal of Machine Learning Research, I1(Jan):171-23 2010.
Moustafa Alzantot, Yash Sharma, Ahmed Elgohary, Bo-Jhang Ho, Mani Srivastava, and Kai- Kai-Wei Chang. Generating natural language adversarial examples. arXiv preprint wrXiv:184.007990 2018.
Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant risk minimization. arXiv preprint arXiv: met::0:00009999 2019.
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give a false sense curity: Circumventing defenses to adversarial examples. In International Conference on Machine Learning, pp. 274-283, 2018.
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In Security and Privacy (SP), 2017 IEEE Symposium on, pp. 39-57. IEEE, 2017a.
Nicholas Carlini and David Wagner. Adversarial examples not easily detected: Bypassing ten detection methods. In Proceedings of the 10th ACM Workshop on Artificial Intelligence and Securify, pp. 3-14. ACM, 2017b.
Nicholas Carlini and David Wagner. Audio adversarial examples: Targeted attacks on speech-to- text. arXiv preprint 1801.0140 2018
Tian Qi Chen, Xuechen Li, Roger B Grosse, and David K Duvenaud. Isolating sources of disentan- glement in variational autoencoders. In Advances in Neural Information Processing Systems, pp. 2620- 2018.
Gamaleldin FF Elsayed, Shreya Shankar, Brian Cheung, Nicolas Papernot, Alex Kurakin, Ian Good- fellow, and Jascha Sohl-Dickstein. Adversarial examples that fool both human and computer vision. arXiv preprint 1802.0889000095 10, 2018.
Logan Engstrom, Brandon Tran, Dimitris Tsipras, Ludwig Schmidt, and Aleksander Madry. Ex- ploring the landscape of spatial robustness. In International Conference on Machine Learning, pp. 1802-1811, 2019.
William T Freeman. The generic viewpoint assumption in a framework for visual perception. Na- fure, 368(6411):542 1994.
Mingming Gong, Kun Zhang, Tongliang Liu, Dacheng Tao, Clark Glymour, and Bernhard Schölkopf. Domain adaptation with conditional transferable components. In International con- ference on machine learning, pp. 2839-2848, 2016.
lan J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6573 2014.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In International Conference on Learning Representations, 2015.
Alison Gopnik, Clark Glymour, David M Sobel, Laura Schulz, Tamar Kushnir, and David Danks. A theory of causal learning in children: causal maps and bayes nets. Psychological review, 111 (1):3, 2004.
Christina Heinze-Deml and Nicolai Meinshausen. Conditional variance penalties and domain shift robustness. arXiv preprint arXiv: / 00. /469, 2017.
Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. beta-vae: Learning basic visual concepts with constrained variational framework. In International Conference on Learning Representations, volume 3, 2017.
10

---

Under review as a conference paper at ICLR 2020
Niki Kilbertus, Giambattista Parascandolo, and Bernhard Schölkopf. Generalization in anti-cause learning. arXiv preprint arXiv:18/2.00524, 2018.
Hyunjik Kim and Andriy Mnih. Disentangling by factorising. In International Conference on Machine Learning, pp. 2654 2663, 2018.
Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv:1312.6114, 2013.
arXiv preprint
Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial examples in the physical world. arXiv preprint 1rX1v:1607.000 00.6000...2,,..... 2016.
Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple unified framework for detecting out-of-distribution samples and adversarial attacks. Advances in Neural Information Process- ing Systems, pp- 7167-7177, 2018.
Yingzhen Li and Stephan Mandt. Disentangled sequential autoencoder. In International Conference on Machine Learning. pp. 5656-5665, 2018.
Yingzhen Li, John Bradshaw, and Yash Sharma. Are generative classifiers more robust to adversarial attacks? arXiv preprint arXiv: 180..0652.... 2018.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint (eKiv:770600600 2017.
Siddharth Narayanaswamy, T Brooks Paige, Jan-Willem Van de Meent, Alban Desmaison, Noah Goodman, Pushmeet Kohli, Frank Wood, and Philip Torr. Learning disentangled representations with semi-supervised deep generative models. In Advances in Neural Information Processing Systems, pp. 5925 59355,5932222 2017.
Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z Berkay Celik, and Ananthram Swami. Practical black-box attacks against machine learning. In Proceedings of the 2017 ACM on Asia Conference on Computer and Communications Security, pp. 506-519. ACM, 2017.
Nicolas Papernot, Fartash Faghri, Nicholas Carlini, Ian Goodfellow, Reuben Feinman, Alexey Ku- rakin, Cihang Xie, Yash Sharma, Tom Brown, Aurko Roy, Alexander Matyasko, Vahid Behzadan, Karen Hambardzumyan, Zhishuai Zhang, Yi-Lin Juang, Zhi Li, Ryan Sheatsley, Abhibhay Garg, Jonathan Uesato, Willi Gierke, Yinpeng Dong, David Berthelot, Paul Hendricks, Jonas Rauber, and Rujun Long. Technical report on the cleverhans v2.1.0 adversarial examples library. arXiv preprint arXiv:1610.00770 2018
Giambattista Parascandolo, Niki Kilbertus, Mateo Rojas- CorullCa and Bernhard Schölkopf. Learning independent causal mechanisms. arXiv preprint arXiv: 2017.
Judea Pearl. Causality. Cambridge university press, 2009.
Judea Pearl and Dana Mackenzie. The book of why: the new science of cause and effect. Basic Books, 2018
Luis Perez and Jason Wang. The effectiveness of data augmentation in image classification using deep learning. arXiv preprint arXiv: 1712.0/71 2017.
Jonas Peters, Dominik Janzing, and Bernhard Schölkopf. Elements of causal inference: foundations foundatie and learning algorithms. MIT press, 2017.
Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. Stochastic backpropagation and approximate inference in deep generative models. arXiv preprint arXiv: 2014.
Dominik Rothenhäusler, Nicolai Meinshausen, Peter Bühlmann, and Jonas Peters. Anchor regres- sion: heterogeneous data meets causality. arXiv preprint arXiv:/801.00 066229, 2018.
Pouya Samangouei. Maya Kabkab, and Rama Chellappa. Defense-Gar Protecting classifiers against adversarial attacks using generative models. In International Conference on Learning Representations, 2018. URL https: http///pppeeeee..cc///ee f

---

Under review as a conference paper at ICLR 2020
Bernhard Schölkopf, Dominik Janzing, Jonas Peters, Eleni Sgouritsa, Kun Zhang, and Joris Mooij. On causal and anticausal learning. arXiv preprint TrXiv:1206.647,, 2012.
Lukas Schott, Jonas Rauber, Matthias Bethge, and Wieland Brendel. Towards the first adversarially robust neural network model MNIST. In International Conference Learning Representa- lions, 2019. URL
.et ftrum? 2orrmmidd-1EEHOSSS5
Yang Song, Taesup Kim, Sebastian Nowozin, Stefano Ermon, and Nate Kushman. Pixeldefend: Leveraging generative models to understand and defend against adversarial examples. Interna- tional Conference Learning Representations, 2018. URL https: http://eeeeeeeeeett/./// Forum?id=rJUYer
Peter Spirtes, Clark N Glymour, Richard Scheines, David Heckerman, Christopher Meek, Gregory Cooper, and Thomas Richardson. Causation, prediction, and search. MIT press, 2000.
Petar Stojanov, Mingming Gong, Jaime Carbonell, and Kun Zhang. Data-driven approach to multiple-source domain adaptation. In The 22nd International Conference on Artificial Intelli- gence and Statistics, pp. 3487-3496, 2019.
Christian Szegedy, Wojciech Zaremba, llya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199, 2013.
Florian Tramèr, Alexey Kurakin, Nicolas Paperot, Ian Goodfellow, Dan Boneh, and Patrick McDaniel. Ensemble adversarial training: Attacks and defenses. In International Confer- ence Learning Representations, 2018. URL https: htttt:::://peerevieee ew. net/ forum? id== rkZvSe-R2. RZ.
Cheng Zhang, Judith Butepage, Hedvig Kjellstrom, and Stephan Mandt. Advances in variational inference. IEFE transactions on pattern analysis and machine intelligence, 2018.
Kun Zhang, Bernhard Schölkopf, Krikamol Muandet, and Zhikun Wang. Domain adaptation under target and conditional shift. In International Conference on Machine Learning, pp. 819-821] 2013.
Han Zhao, Remi Tachet des Combes, Kun Zhang, and Geoffrey J Gordon. On learning invariant representation for domain adaptation. arXiv preprint arXiv: 190,0090,, 2019.
12

---

Under review as a conference paper at ICLR 2020
A DERIVATION DETAILS
A. THE INTERVENTION ELBO
When training with clean data D = {(In, }. we set the manipulation variable M to a null value, e.g. do(m 0). In this case would like to maximise the log- lggliikeidhodd of the intervened model, ie.
max Epllo po yldo(m 0))}
This log-likelihood of the intervened model is defined by integrating out the unobserved latent vari- able Z in the intervened joint distribution, and from do-calculus we have
log Po,, yldo(m 0)) log / po v. 2|d0(m 0))
(12)
= log /
pa mm = 0)p(y)p(z)dz.
variational lower-bound (or ELBO) of the log- likelihhoddd uses variational distribution ((=l-)
logpa g/domm = 0)) = log / po (e m 0)p())p(2))))))))))))))))))))))) g(x)-) dz
((2
(13)
2 (g(+))) log po(xly m 0)p(y)p(z)
g(=l.)
The lower-bound holds for arbitrary ((=l-) as long as it is absolutely continuous w.r.t. the posterior distribution (= xr y, do(m = 0)) of the intervened model. Now recall the design of the inference network/variational distribution in the main text:
(2, m|x, y) = 4o, (= jr, y, m Jop (m|(r),
where Φ1 and 6h are the inference network parameters the corresponding variational distributions. Performing intervention do(m 0) on this q distribution gives
g a. y. do(m 0)) gøn (z|æ, y, 0).
Defining g(2l-) 4o (z|z, y, do(m 0)) and plugging-in in it to eq. (13) return the intervention
ELBO objective (3) presented in the main text.
A.2 THE ELBO FOR UNLABELLED TEST DATA
The proposed fine-tuning method in the main text require optimising the marginal log- ligeliihd log (2)) for æ ~ D. which clearly intractable. Instead of using a variational distribution for the unobserved class label Y, we consider the variational lower-bound of log Pe (z, y) for all possible
y = Ve:
lo v) log po v, m))zdm
log /
90(2, m m| y)
po N, m)
dzdm
(14)
(2 m m|x, y)
log Po (x. 2, m)
= ELBO(x, v)-
(2 m v)
Since both logarithm and exponent functions preserve monotonicity, and for all ye, c 1, I,..., C we have log (e yc) > ELBO(x, ye). we have
log (o yo) 2 ELBO(z, No), Vc
+ po(x, vo) 2 exp|ELBO(r, yo)], Vc
log = log
Σ pa vo)
> log Σ exp|EEBBOxz,) yo)] ELBO(r),
which justifies the ELBO objective (7) defined in the main text.
13

---

Under review as a conference paper at ICLR 2020
B ADDITIONAL RESULTS
CNN We also performed experiments using different DNN network architectures. The convo- lution layers in CNN designed to be robust to shifts. Thus, test these vertical and hori- zontal shifts with a standard CNN architecture as used in https: /ppps://keras.eee io/ wt/pooapees/////////// cifari0_enn/ 4 convolution layers used in this architecture.
Figure 16 shows the performance against different shifts. We see that adding vertical shifts to the training data clearly harmed the robustness performances to unseen horizontal shifts as shown in 17(b). Adding horizontal shifted images in training did not influences the performance on vertical shifts much. Thus, we see that using different architectures of DNN, even the one that are designed to be robust to these manipulations, lack of generalization ability to unseen data is common problem.
1.0
0.9
0.8
0.8
0.7
0.7
0.6
0.6
0.4
train clean test
0.4
train clean test ht
0.3
train ht test vt
0.3
train test ht
0.2
vt vt
0.2
0.1
0.1
0.0
0.1 0.2 0.4 0.5 0.6 0.7 0.8 0.9
0.0
0.1 0.3 0.4 0.5 0.6 0.7 0.8 0.9
Vertical shift range
Horizontal shift range
(a) Test Vertical shift
(b) Test Horizontal shift
Figure 16: Robustness results for DNNs against different manipulations on MNIST using CNN. Panels (a) and (b) show the accuracy on classifying noisy test data generated by shifting the digits vertically (vt) and horizontally (ht). It shows that data augmentation during training makes general- ization to unseen shifts worse (orange versus blue lines).
1.0
1.0
0.9
0.9
Accurac
0.7
Train Clean Test Ver
0.7
Train Clean Test Hor
0.6
Train Hor Test Ver
0.6
Train Ver Test Har
0.5
Train Ver test Ver
3 0.5
Train Hor Test Hor
0.4
0.3
0.2
0.2
0.1
0.1
0.0
0.1 0.2 0.4 0.6 0.7 0.8 0.9
0.0
0.1 0.2
0.3
0.5 0.7 0.8 0.9
Vertical shift range
Horizontal shift range
(a) Test Vertical shift
(b) Test Horizontal shift
Figure 17: Robustness results for DNNs against different manipulations MNIST using large MLP. Panels (a) and (b) show the accuracy on classifying noisy test data generated by shifting the digits vertically (yt) and horizontally (ht). It shows that data augmentation during training makes generalization to unseen shifts worse (orange versus blue lines).
Enlarge Network Size Here we exam whether network capacity has any influence on the robust- ness performance to unseen manipulation. We use a wider network with [1024, 512, 512, 1024] units in each hidden layer instead of [512, 256, 126, 512] sized network in the paper. Figure 17 shows the robustness performance using this enlarged network. We observe the similar degree of over-fitting to the augmented data. The penalization ability shows no improvement by enlarging the network sizes.
ZCA Whitening Manipulation Our result does not limited to shifts, it generalizes to other ma- nipulations. Figure 18 compare the result from training with clean images and training with ZCA whitening images added. We see that adding ZCA whitening images in training harm both robust- ness against vertical shift and horizontal shift.
14

---

Under review as a conference paper at ICLR 2020
1.0
1.0
0.9
0.9
0.8
0.8
0.6
0.6
0.5
0.5
0.4
0.4
train clean test
train clean test ht
0.2
train zca vt
0.2
train zca test ht
0.1
0.0
0.1 0.2 0.4 0.6 0.7 0.8 0.9
0.1
0.0
0.1 0.2 0.3 0.5 0.7 0.8 0.9
Vertical shift range
Horizontal shift range
(a) Test Vertical shift
(b) Test Horizontal shift
Figure 18: ZCA Whitening manipulation result. Figure shows the robustness results for DNNs against different manipulations on MNIST using CNN. The blue curve shows that result from train- ing with clean data. The orange curve shows that result from training with zea whitening data added.
1.0
1.0
0.9
0.25
0.9
Accuracy cv
0.8
0.01
Accuracy
0.8
0.7
0.05
0.75
0.7
0.05
0.8
0.1
1.0
0.1
1.0
0.6
0.5
0.5
0.4
0.3
0.3
0.2
0.2
0.0 0.1 0.2 0.4 0.6 0.7 0.8 0.9
0.1 0.2 0.3 0.6 0.8 0.9
Vertical shift range
Vertical shift range
Figure 19: Performance regarding different percentage of test data used for fine-tuning manipulation of horizontal shift without us- ing do(m) = 0 for the cleaning training data during -uning.
Figure 20: Performance regarding different percentage of test data used for fine-tuning manipulation of vertical shift using do(m) 0 for the cleaning training data during fine- tuning.
Additional Figures In addition to Figure 8, We also show the result testing with Vertical shift show in Figure 19, where smaller No network ([dimM, 500, 500] is used. The conclusion is the same was using the vertical shift. We need very few data for fine-tune. More than 1% data is sufficient.
Similar as Figure 8, we show the result using different percentage of data for fine-tuning in this experiment setting in 20.
C EXPERIMENTAL SETTINGS
Network architecture
MNIST experiments:
Discriminative DNN: The discriminate model used in the paper contains densely
connected hidden layer [512, 256, 126, 512] width for each layer. ReLU activations
and dropout are used with dropout rate [0.25, 0.25, 0.25, 0.5) for each layer.
Deep CAMA's p networks: we use dim
10, dim Z)) = 64 and dim 32.
NNE MLP of layer sizes [dim(Y). 500, 500) and ReLU activations.
NN an MLP of layer sizes (dim(2), 500, 500 and ReLU activations.
N A MLP of layer sizes [dim(M), 500, 500, 500, 500) and ReLU activations.
NN
merge an projection layer which projects the feature outputs from the previous
networks to a 3D tensor of shape (4, 4, 64), followed by deconvolutional layers with
stride 2, SAME padding, filter size (3, 64, 64) except for the last layer (3, 3, 64, 1).
15

---

Under review as a conference paper at ICLR 2020
All the layers use ReLU activations except for the last layer, which uses sigmoid activation.
Deep CAMA's g networks:
NN : it starts from a convolutional neural network (CNN) with 3 blocks of (conv3 x 3, max-pool] layers with output channel size 64, stride 1 and SAME padding, then performs a reshape-to-vector operation and transforms this vector with an MLP of layer sizes [4 x 64, 500, dim(M M) 2] to generate the mean and log-variance of ((m[xz) All the layers use ReI.U activation except for the last layer, which uses linear activation.
NN3: first it uses CNN with similar architecture as NN CNN (except that the filter size is 5) to process z. Then after the reshape-to- to-vtor operation, the vector first gets transformed by an MLP of size [4 x 4 64, 500], then it gets combined with and m and passed through another MLP of size [500+dimm(Y] dimm M), 500, dim( Z xxx) to obtain the mean and log-variance of g(=|2r, y, m). All the layers use ReLU activa- tion except for the last layer, which uses linear activation.
Measurement data experiments:
Discriminative DNN:
The A, C, variables are concatenated an input vector of total dimension 20. Then the DNN contains densely connected hidden layer of [64, 16, width for each layer, and output Y. ReLU activations and dropout are used with dropout rate [0.25, 0.25, 0.5] for each layer.
Deep CAMA's p networks: we use dim(Y) 5, dA)) 5, dimmm Z) = 64 and (i) = 32.
5, dim(C)
p(yla): an MLP of layer sizes [dim(A), 500, dim)) ReLU activations except for the last layer (softmax).
p(xly, C, 2, m) contains 5 networks: 4 networks {NNY, NN NN 2, NN to pro- cess each of the parents of X, followed by merging network.
NNC an MLP of layer sizes (dim()),500)) 500, 500) and ReLU activations.
NNÉ.. MLP of layer sizes [dim(C), 500, 500] and ReLU activations.
NN an MLP of layer sizes (dim(2,,50)) 500) and ReLU activations.
NN 1 MLP of layer sizes [dim(M/), 500, 500, 500, 500) and ReLU activations.
NN merge: it first start from a concatenation of the feature outputs from the above
4 networks, then transforms the concatenated vector with an MLP of layer sizes [500 x 4, 500, dim(X) to output the mean of x. All the layers use ReLU activations except for the last layer, which uses linear activation.
Deep CAMA's networks:
(mm[x): it uses an MLP of layer sizes [dim(X), 500, 500, dim(M) x 2] to obtain the mean and log-variance. All the layers use ReLU activations except for the last layer, which uses linear activation.
(=2/1, y, m, a, c): it first concatenates y, m, a, into a vecto, then uses an MIP of layer sizes [dim(X) + dim(Y) + dim(M) ()+++ + dim(A) (A) + dim(C") 500, 500, dim( [) x 2] to transform this vector into the mean and log- variance of q(2|2, y, m, a, c). All the layers use ReLU activations except for the last layer, which uses linear activation.
ClnaR-binary experiments:
Discriminative CNN:
The discriminate model used in the paper is a CNN with 3 con- volutional layers of filter width 3 and channel sizes [128. 128, 128], followed by flattening operation and a 2-hidden layer MI.P of size [4 x 128, 1000, 1000, 10). It uses ReL.U activations and max pooling for the convolutional layers.
Deep CAMA's p networks: we use dim(Y)= 10, dim (Z) 128 and dim(M) 64.
NNÉ- an MIPP of layer sizes [dim(Y), 1000, 1000) and ReL.U activations.
N an MIP of layer sizes (dim(Z), 1000, 1000 and ReI.U activations.
N M MLP of layer sizes [dim(M), 1000, 1000, 1000] and ReLU activations.
NN
merge an projection layer which projects the feature outputs from the previous
networks to a 3D tensor of shape (4,4, 64), followed by deconvolutional layers with stride 2, SAME padding, filter size (3, 3, 64, 64) except for the last layer (3, 3, 64,3). All the layers use ReLU activations except for the last layer, which uses sigmoid activation.
16

---

Under review as a conference paper at ICLR 2020
Deep CAMA's g networks:
N it starts from a convolutional neural network (CNN) with 3 blocks of (conv3 x 3, max-pool] layers with output channel size 64, stride 1 and SAME padding, then per- forms reshape-to-vectt
operation and transforms this vector with an MLP of layer sizes [4 x 4 64, 1000, 1000, dim M) to generate the mean and log-variance of (mm[z). All the layers use ReLU activation except for the last layer, which uses linear activation.
2 first it re-uses NN M CNN network for feature extraction on 2. Then after the reshape-to-veet operation, the vector gets combined with y and and passed through another MLP of size [4 4 64 d to obtain the mean and log-variance q(a|>,, y, m). All the layers use ReLU activa- tion except for the last layer, which uses linear activation.
1000, 1000, dim( Z) x
Measurement data generation We set the target to be categorical, its children, co- co-parents and parents are continuous variables. The set classes for Y, and Y has 10 children variables and co-parents variables, also one 5 dimensional parents.
Parents (A) and co-parents (C) generated by sampling from normal distribution. We generate Y using structured equation Y = (A) + ay. We use fy = argmax g(A) and g() is a quadratic function 0.2 2 0.8A. øy is the Gaussain noise.
To generate the children X f(Y, du, also used quadratic function f and the parameters were sampled from a Gaussian distribution. As in the experiment, we were using fixed scale shift, also added normalize the children before adding the Gaussian random noise da- So that all observations are similar scale.
Other For MNIST experiments, we uses 5% of the training data as the validation We used the training results with the highest validation accuracy for testing. If not otherwise specified, 50% of noisy test data are used for fine-tuning in the shift experiments and data are used for fine-tuning in the attack experiments.
For the experiments with measurement data. We generated 1000 data in total. We split, 500 data for testing, 450 for training and 50 for validation. We used the training results with the highest validation accuracy for testing for both deep CAMA and for DNN.
17


Reviews:
[{"review_id": "rkenLPW8tS", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper tackles a very important problem, manipulation robustness of modern machine learning models, by applying a chain of design choices perfectly well:\n  * VAEs as probabilistic generators\n  * Manipulations as independent causes that can be generated by interventions\n  * Causal inference for manipulation-corrected training\n  * Bayesian inference for robust prediction\n\nSticking to the Bayesian point of view, the method can perform model averaging across potential manipulations at the test time. This is an extremely elegant property, which is also proven in the experiments to be very effective.\n\nBeing able to learn previously unseen types of manipulations only by proper application of causal inference tools is a very important news for the adversarial robustness community.\n\nFigure 9 is a spectacular proof of concept to illustrate the disentanglement property of the proposed method.\n\nI have only one point for improvement. I do not buy the argument that we should do q(m|x) but not q(m|x,y). Why should we exclude class-specific manipulations? This wouldn't affect the cause (the class) but only the outcome, so would be a valid manipulation. I would actually expect the model to work still well with q(m|x,y). Could the authors comment on what the concrete benefit of leaving out y from manipulations is?\n\nThe construction of the loss in Eq 5 is in spirit semi-supervised learning on m. We show the true m=0 cases to the model during training but assume not to know the labels of the manipulated samples. It could be beneficial to draw a link to semi-supervised learning here and even tie it to the arguments about the relationship between causal inference and semi-supervised learning:\n\nSchölkopf et all., On Causal and Anticausal Learning, ICML, 2012.\n", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly."}, "extra_text": {}}, {"review_id": "SJgvtlhG9H", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "In this paper, the author first assume the data generation process that sample X is generated by label Y, latent style (domain information) Z and other manipulations M and then propose the deep causal manipulation augmented model that use the do calculus to model the manipulations of data and further take it as the cause to the observed effect variables x. The author further elaborately devises experiment on the standard dataset and achieve good results.\n\nStrong point:\n(1)\tThe paper is well-written and the idea is motivated.\n(2)\tThe experiment is convincing and the result is good.\n\nWeak point:\n(1)\tPlease provide the deducing process of the ELBO, such as how the ELBO is deduced in the training mode and the prediction process should be elaborated detailedly.\n(2)\tThe symbolism is confusing, such as the \\phi_1 and \\phi_2 in Eq(6).\n(3)\tWhat is the difference between manipulations M and the latent style Z? Do they belong to the domain information? Please justify. \n", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "HkeAvEn2iH", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\n(1) The paper makes vague and unsupported claims about causality.\n   The notion of 'causal', as well as do() notation, is really hollowed out here. It is of limited significance. In Equation (3) do() disappears from one line to the next, illustrating how little it does.\n\n(2) Lack of novelty: The paper references \"Learning disentangled representations with Semi-supervised deep generative models\", but fails to mention that their model is is nearly identical. \n\n(3) The quality is on the low side; confusing mistakes in figures, in some\n sections poor proof-reading.\n\nDetails:\n(1) Exaggerated claims.\n\"We argue that the incapability for causal reasoning is the reason of DNN's vulnerability\".  (This is stated twice in different words).\n\n  Unfortunately, the term \"causal reasoning\" is used without making it precise.  The paper has a very weak relation to causality; one could replace \"causal\" with \"translation invariant\", and very little would change.  Their experiments try to learn translation invariance (appendix shows whitening invariance).\n\nThe paper makes gratuitous use of do() notation; for example, regular MNIST is reinterpreted as not being observational, but as arising from an intervention, namely do(translation=0).  But do() does not amount to much here. It reduces to mere conditioning here, as the do is always applied to a root node without parents.  This is clear in equation in 3, where do's are correctly just deleted from one line to the next; do(), and 'causal' are used mostly as a rhetorical device.\n\n(2) The model in the paper is pretty much the same as \"Learning disentangled representations with Semi-supervised deep generative models\", figure 2 here corresponds to figure 4 there.  Both papers use partially supervised (or unsupervised) VAEs. There are some differences, e.g. the prior work considers a more general case and uses importance sampling.  This paper refers but fails to say how similar they are.  There should be a comparison between the difference inference methods used.\n\n\n(3) The paper is sloppy in places.  \n\nFigure 6: the dotted arrow between nodes A and Z should not be there.\nIt does not follow from the graphical model in Figure 5.\n\nIn Figure 12a),  the text contradicts the figure. The brown curve with fine tuning has lower accuracy than the green one without fine tuning, but the text says the opposite. \n\nFigure 11: there is a curve mislabelled as \"Dis\"\n\nResults are mixed.  While results on vertical translation invariance and adversarial robustness seem good, the method fails to achieve horizontal translation invariance (Figure 7 b) and c)) - disappointing. But some of the translation invariance seems to be due to just using a generative model (the VAE), as it occurs without any fine-tuning.\n", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper attempts to present a causal view of robustness in classifiers, which is a very important area of research.
However, the connection to causality with the presented model is very thin and, in fact, mathematically unnecessary. Interventions are only applied to root nodes (as pointed out by R4) so they just amount to standard conditioning on the variable "M". The experimental results could be obtained without any mention to causal interventions.
Decision: Reject