Alleviating Privacy Attacks via Causal Learning

1 INTRODUCTION . Machine learning algorithms , especially deep neural networks ( DNNs ) have found diverse applications in various fields such as healthcare ( Esteva et al. , 2019 ) , gaming ( Mnih et al. , 2013 ) , and finance ( Tsantekidis et al. , 2017 ; Fischer & Krauss , 2018 ) . However , a line of recent research has shown that deep learning algorithms are susceptible to privacy attacks that leak information about the training dataset ( Fredrikson et al. , 2015 ; Rahman et al. , 2018 ; Song & Shmatikov , 2018 ; Hayes et al. , 2017 ) . Particularly , one such attack called membership inference reveals whether a particular data sample was present in the training dataset ( Shokri et al. , 2017 ) . The privacy risks due to membership inference elevate when the DNNs are trained on sensitive data such as in healthcare applications . For example , patients providing medical records to build a model that detects HIV would not want to reveal their participation in the training dataset . Membership inference attacks are shown to exploit overfitting of the model on the training dataset ( Yeom et al. , 2018 ) . Existing defenses propose the use of generalization techniques such as adding learning rate decay , dropout or using adversarial regularization techniques ( Nasr et al. , 2018b ; Salem et al. , 2018 ) . All these approaches assume that the test data is from the same distribution as the training dataset . In practice , a model trained using data from one distribution is often used on a ( slightly ) different distribution . For example , hospitals in one region may train a model to detect HIV and share it with hospitals in different regions . However , generalizing to a new context is a challenge for any machine learning model . We extend the scope of membership privacy to different distributions and show that the risk from membership attack increases further on DNNs as the test distribution is changed . That is , the abiltity of an adversary to distinguish a member from a non-member improves with change in test distributions . To alleviate privacy attacks , we propose using models that depend on the causal relationship between input features and the output . Causal learning has been extensively used to guarantee fairness and explainability properties of the predicted output ( Kusner et al. , 2017 ; Nabi & Shpitser , 2018 ; Datta et al. , 2016 ) . However , the connection of causal learning to privacy is yet unexplored . To the best of our knowledge , we provide the first analysis of privacy benefits of causal models . By definition , causal relationships are invariant across input distributions ( Peters et al. , 2016 ) , and hence make the predictions of causal models independent of the observed data distribution , let alone the observed dataset . Hence , causal models generalize better even with change in the distributions . In this paper , we show that the generalizability property of causal models directly ensures better privacy guarantees for the input data . Concretely , we prove that with reasonable assumptions , a causal model always provides stronger ( i.e. , smaller value ) differential privacy guarantees than a corresponding associational model trained on the same features and the same amount of added noise to the training dataset . Consequently , we show that membership inference attacks are ineffective ( equivalent to a random guess ) on causal models trained on infinite samples . Empirical attack accuracies on four different datasets confirm our theoretical claims . We find that 60K training samples are sufficient to reduce the attack accuracy of a causal model to a random guess . In contrast , membership attack accuracy for neural network-based associational models increase as test distributions are changed . The attack accuracy reaches nearly 80 % when the target associational model is trained on 60K training samples and used to predict test data that belong to a different distribution than the training data . Our results show that causal learning approaches are a promising direction for training models on sensitive data . Section 2 describes the properties of causal models . Section 3 proves the connections of causality to differential privacy and robustness to membership attacks . Section 4 provides empirical results . To summarize , we make the following contributions : • For the same amount of added noise , models learned using causal structure provide stronger -differential privacy guarantees than corresponding associational models . • Causal models are provably more robust to membership inference attacks than typical associational models such as neural networks . • We simulate practical settings where the test distribution may not be the same as the training distribution and find that the membership inference attack accuracy of causal models is close to a “ random guess ” ( i.e. , 50 % ) while associational models exhibit upto 80 % attack accuracy . 2 PROPERTIES OF CAUSAL MODELS . Causal models are shown to generalize well since the output of these models depend only on the causal relationship between the input features and the outcomes instead of the associations between them . From prior work , we know that the causal relationship between the features is invariant to the their distribution ( Peters et al. , 2016 ) . Using this property , we study its effects on the privacy of data . 2.1 BACKGROUND : CAUSAL MODEL . Intuitively , a causal model identifies a subset of features that have a causal relationship with the outcome and learns a function from the subset to the outcome . To construct a causal model , one may use a structural causal graph based on domain knowledge that defines causal features as parents of the outcome under the graph . Alternatively , one may exploit the strong relevance property from Pellet & Elisseeff ( 2008 ) , use score-based learning algorithms ( Scutari , 2009 ) or recent methods for learning invariant relationships from training datasets from different distributions ( Peters et al. , 2016 ; Bengio et al. , 2019 ) , or learn based on a combination of randomized experiments and observed data . Note that this is different from training probabilistic graphical models , wherein an edge conveys an associational relationship . Further details on causal models are in Pearl ( 2009 ) ; Peters et al . ( 2017 ) . For ease of exposition , we assume the structural causal graph framework throughout . Consider data from a distribution ( X , Y ) ∼ P where X is a k-dimensional vector and Y ∈ { 0 , 1 } . Our goal is to learn a function h ( X ) that predicts Y . Figure 1 shows causal graphs that denote the different relationships between X and Y . Nodes of the graph represent variables and a directed edge represents a direct causal relationship from a source to target node . Denote Xpa ⊆ X , the parents of Y in the causal graph . Figure 1a shows the scenario where X contains variables XS0 that are correlated to Xpa in P , but not necessarily connected to either Xpa or Y . These correlations may change in the future , thus a generalizable model should not include these features . Similarly , Figure 1b shows parents and children of Xpa . The d-separation principle states that a node is independent of its ancestors conditioned on all its parents ( Pearl , 2009 ) . Thus , Y is independent of XS1 and XS2 conditional on Xpa . Therefore , including them in a model does not add predictive value ( and further , avoids problems when the relationships between XS1 and XS2 may also change ) . Finally , for completeness , the exhaustive set of variables to include is known as the causal Markov Blanket1 , XC which includes Y ’ s parents , ( Xpa ) , children ( Ych ) and parents of children . Conditioned on its Markov blanket ( Figure 1c ) , Y is independent of all other variables in the causal graph . When Y has no descendants in the graph , then the effective Markov blanket includes only its parents , Xpa . The key insight is that building a model for predicting Y using the Markov Blanket XC ensures that the model generalizes to other distributions of X , and also to changes in other causal relationships between X , as long as the causal relationship of XC to Y is stable . We call such a model as a causal model , the features in ( XC ) as the causal features , and assume that all the causal features for Y are observed . In contrast , we call a model that uses all available features as an associational model . 2.2 GENERALIZATION TO NEW DISTRIBUTIONS . We state the generalization property of causal models and show how it results in a stronger differential privacy guarantee . We first define In-distribution and Out-of-distribution generalization error . Throughout , L ( . , . ) refers to the loss on a single input and LP ( . , . ) = EPL ( . , . ) refers to the expected value of the loss over a distribution P ( X , Y ) . We refer f : X→ Y as the ground-truth labeling function and h : X→ Y as the hypothesis function or simply the model . Then , L ( h , h′ ) is any loss function quantifying the difference between two models h and h′ . Definition 1 . In-Distribution Generalization Error ( IDE ) . Consider a dataset S ∼ P ( X , Y ) . Then for a model h : X→ Y trained on S , the in-distribution generalization error is given by : IDEP ( h , f ) = LP ( h , f ) − LS∼P ( h , f ) ( 1 ) Definition 2 . Out-of-Distribution Generalization Error ( ODE ) . Consider a dataset S sampled from a distribution P ( X , Y ) . Then for a model h : X→ Y trained on S , the out-of-distribution generalization error with respect to another distribution P∗ ( X , Y ) is given by : ODEP , P∗ ( h , f ) = LP∗ ( h , f ) − LS∼P ( h , f ) ( 2 ) Definition 3 . Discrepancy Distance ( discL ) ( Def . 4 in Mansour et al . ( 2009 ) ) . Let H be a set of hypotheses , h : X→ Y . Let L : Y× Y→ R+ define a loss function over Y for any such hypothesis h. Then the discrepancy distance discL over any two distributions P ( X , Y ) and P∗ ( X , Y ) is given by : discL ( P , P ∗ ) = max h , h′∈H |LP ( h , h′ ) − LP∗ ( h , h′ ) | ( 3 ) Intuitively , the term discL ( P , P∗ ) denotes the distance between the two distributions . Higher the distance , higher is the chance of an error when transferring h from one distribution to another . Now , we will state the theorem on the generalization property of causal models . Theorem 1 . Consider a structural causal graph G that connects X to Y , and causal features XC where XC is a Markov Blanket of Y under G. Let P ( X , Y ) and P∗ ( X , Y ) be two distributions with arbitrary P ( X ) and P∗ ( X ) such that the causal relationship between XC and Y is preserved , which implies that P ( Y|XC ) = P∗ ( Y|XC ) . Let f : XC → Y be the resultant invariant labelling function such that y = f ( XC ) . Further , assume thatHC represents the set of causal models hc : XC → Y that use all causal features andHA represent the set of associational models ha : X→ Y that may use all available features , such thatHC ⊆ HA and f ∈ HC . Then , for any symmetric loss function L that obeys the triangle inequality , the upper bound of ODE from a dataset S ∼ P ( X , Y ) to P∗ ( called ODE-Bound ) for a causal model hc ∈ HC is less than or equal to the upper bound ODE-Bound of an associational model ha ∈ HA , with probability at least ( 1− δ ) 2 . ODE-BoundP , P∗ ( hc , f ; δ ) ≤ ODE-BoundP , P∗ ( ha , f ; δ ) ( 4 ) 1We call it the causal Markov Blanket since it is based on the structural causal graph , to distinguish it from the associational Markov Blanket that is based on conditional probability distribution from a Bayesian network . Proof . As an example , consider a colored MNIST data distribution P where the classification task is to detect whether a digit is greater than 5 , and where all digits above 5 are colored with the same color . Then , under a suitably expressive class of models , the loss-minimizing associational model may use only the color feature to obtain zero error , while the loss-minimizing causal model will still use the shape ( causal ) features to obtain zero error . On any new P ∗ that does not follow the same correlation of digits with color , we expect that the loss-minimizing associational model will have higher error than the loss-minimizing causal model . Formally , since P ( Y|XC ) = P∗ ( Y|XC ) , the optimal causal model that minimizes loss over P is the same as the loss-minimizing model over P∗ . That is , hOPTc , P = h OPT c , P∗ . However , for some associational models , hOPTa , P 6= hOPTa , P∗ and thus there is an additional loss term when generalizing to data from P ∗ . The rest of the proof follows from triangle inequality of the loss function and the standard bounds for IDE from past work . Detailed proof is in Appendix Section A.1 . Corollary 1 . Consider a causal model hc : XC → Y and an associational model ha : X→ Y trained on a dataset S ∼ P. Let ( x , y ) ∈ S and ( x′ , y′ ) /∈ S be two input instances such that they share the same true labelling function y = f ( xc ) and y′ = f ( x′c ) . Then , the worst-case generalization error for a causal model on any such x′ is less than or equal to that for an associational model . [ Proof in Appendix Section A.2 ] max x∈S , x′ : y′=f ( x′c ) Lx′ ( hc , f ) − Lx ( hc , f ) ≤ max x∈S , x′ : y′=f ( x′c ) Lx′ ( ha , f ) − Lx ( ha , f ) ( 5 )

Summary:    The authors consider a transfer learning problem where the source distribution is P(X,Y) while the target distribution is P*(X,Y) and classifier is trained on data from the source distribution. They also assume that the causal graph generating the data (X and Y) is identical while the conditional probabilities (mechanisms) could change between the source and the target. Further, they assume that if X_C is the Markov Blanket for variable Y in P and P*,  then P(Y|X_C) = P*(Y|X_C). Therefore the best predictor in terms of cross entropy loss for both distributions is identical if it focuses on the variables in the Markov Blanket. Authors define "causal hypothesis" as the one that uses only variables in the Markov Blanket (X_C) to predict Y.