Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 3: Weak Reject
review_content: Summary:

   The authors consider a transfer learning problem where the source distribution is P(X,Y) while the target distribution is P*(X,Y) and classifier is trained on data from the source distribution. They also assume that the causal graph generating the data (X and Y) is identical while the conditional probabilities (mechanisms) could change between the source and the target. Further, they assume that if X_C is the Markov Blanket for variable Y in P and P*,  then P(Y|X_C) = P*(Y|X_C). Therefore the best predictor in terms of cross entropy loss for both distributions is identical if it focuses on the variables in the Markov Blanket. Authors define "causal hypothesis" as the one that uses only variables in the Markov Blanket (X_C) to predict Y.

 In this setting, the authors show two sets of results: a) Out of Distribution Generalization Error is less for the optimal causal predictors from any class of causal hypotheses under any loss function. b) If we tweak the definition of differential privacy where neighboring datasets are defined by replacing one of the variables in the training set by a sample from the test distribution (I have lots of questions about this definition later on), then the authors show that causal classifiers that depend only on the Markov Blanket has lower sensitivity to the change than that of associative classifiers (that use all features). This is used to prove that the causal ones have tighter differential privacy guarantees than the associative ones
 c) Using the differential privacy results, they also show that optimal causal classifiers are more resistant to membership attacks.


The authors demonstrate the results through membership attack accuracies on causally trained and associative models on 4 datasets where the causal graph and the parameters (conditional probabilities) of the Bayesian Network are known apriori.


Major Issues:
    I have lots of issues with the theory in the paper. Thats the main reason for my recommendation.

  1. Why is h_{c,P}^{OPT} = h_{c,P*}^{OPT} ?? (Equation 23 in Page 11) - Authors say that since the markov blanket is the only thing used to predict Y for causal predictors and P (Y|X_C) = P *(Y|X_C), this should be true. But I have problems with this statement/argument. First of all, this is claimed for for any loss function L (not just Cross Entropy Loss) and particularly for a generic hypothesis class H_C (that depends only on Markov Blanket).  

Consider the following counter example - Suppose all the features are in the Markov Blanket of Y (even a simpler case where all features are causal parents of Y). Suppose the true P (Y|X) is a logistic model with weights w_1 on one part of the domain D_1 and with weights w_2 for another part of the domain D_2. Suppose P is a mixture distribution of D_1 and D_2. P* is another mixture distribution (mixed differently) of D_1 and D_2.

Suppose I consider the class of logistic classifiers (but a single logistic model with one weight w) to be my hypothesis class and I use the standard logistic loss (logistic regression) on P, since a single logistic model with one slope cannot match the different slopes in different parts of the domain, it will result in some weight vector w^{opt}_1. Now, since the mixtures of D_1 and D_2 are different in the P* (but P (Y|X ) is identical), the optimal w^{opt}_2 for the P* will be different. 

So for arbitrary hypothesis classes (that do not capture the true P(Y|X_c)) and for a non cross entropy loss - clearly this does not hold at all !! Covariate shifts amongst X_C alone with create a different classifier for an arbitrary loss (even if P (Y|X_C) is the same across both). In fact, the only way I see to salvage this is to assume Cross Entropy loss and talk about all soft classifiers (without restrictions to hypothesis class). But even if thats the case, then the best associational model will be the one that uses the Markov Blanket too ! .

This claim about h_C is crucially used in proof of theorem 1 (Page 11) and Proof of Corollary 1 (Page 13). This is fundamental to all theorems later. That brings into question the validity of many (if not all) the theoretical results in the paper. Authors must address this. 

2. Issues regarding definition of certain quantities.  

a) In equation 5 a quantity max_{x,x'} L_{x sampled from P} (h_c,f) - L{x' sampled from P*} (h_c,f) is defined - the inner quantity is a random quantity that depends on the samples x and x', Then what is the max operator over ?? - What does it mean to have worst case over samples from a distribution ??? Does it mean samples from two different domains ?? 
Even the quantity does not seem to be well defined. 

b) Similar issue occurs in Lemma 1 - Neighboring datasets S and S' are created by first sampling S from P and then S' is obtained by replacing an arbitrary point in S by a random point from P*. Then sensitivity is defines as a max over pair of neighboring datasets - again S and S' are random samples, so what is the max over ?? If it is the worst case - why is the sampling coming in there ? Since it uses Corollary 1 - the main result inherits the same fundamental issues the has been pointed out above.

c) In theorem 2, {cal F}_a is an algorithm. What does it mean to add noise ? - Does it mean you add noise to the model parameters ?? - This is confusing at best.

Minor Issues:
1. Authors claim that connection between causality and privacy has not been explored (Page 2). Pls refer to https://arxiv.org/pdf/1710.05899.pdf where differential privacy itself is related to interventional effects in a system. This connection is very different from the scope of the current paper. However, the statement by the authors is strictly not true.

2. Why is the ground truth function f:X->Y (Section 2.2.) relevant when clearly you have distribution P (X,Y) and P*(X,Y) ?? We might as well define Loss with L (h(x), y) where (x,y) is drawn according to P. What f is never defined anywhere. Authors seems to mean the suggestion I just said in the paper. Authors could clarify. This confuses stuff in the proofs too.

3. Markov Blanket is not causal by any means in my opinion. It is just a minimal set of features conditioned on which Y does not depend on anything else. This only requires conditional independence tests to determine - a purely observational notion - in fact the markov blanket only depend on the moralized graph which does not change across the members of the equivalence class. So calling it causal is a bit confusing. If the features referred to least causal Parents - then still it would be consistent with the invariance in the Invariant Causal Prediction Literature (Peters et al 2016.) and would be causal.








review_title: Official Blind Review #2

final_score: rating: 3: Weak Reject
review_content: The paper proposes using causal learning models for alleviating privacy attacks, i.e. membership inference attacks. The paper proves that causal models trained on sufficiently large samples are robust to membership inference attacks; they confirm the theories with experiments on 4 synthetic data.
The paper is well written; theoretical proof seems correct as it combines proof of differential privacy guarantees  in Papernot et al. 2017, robustness to membership attacks in Yeom et al. 2018 with the generalization property of causal models from Pearl 2009 and Peters et al. 2017. Results are presented clearly. The paper is novel as the authors claimed they provide the first analysis of privacy benefits of causal models.
The main concern of this paper is the results are only confirmed on synthetic data, where all the 4 datasets are generated from known Bayesian networks (i.e., causal graphs). It doesn’t matter if these Bayesian nets are complex or not, because most of the experiments are done with the known true causal models except the last experiment in Figure 3c. Even with learnt causal models, they were learning a Bayesian net from too optimistic data that were indeed generated from Bayesian nets, but these are usually not true for real world data. So evaluations on real dataset, or other synthetic data that are not generated from Bayesian nets are necessary for validating the methods.
Another question is about the ‘causal models are known to be invariant to the training distribution and hence generalize well to shifts between samples from the same distribution and across different distributions.’  More explanations about ‘invariance’ is needed. For example, in Figure 2a and Figure 3a, causal models have similar performance (except Alarm data) with DNN models on test 2, where test samples are generated from different distributions than training samples. Also in Figure 3b, the attack accuracy are no different between causal models and DNN on test 1.
The last minor question is why only parents of Y are included in causal models in the experiments, but not the Markov blanket as stated earlier in Figure 1. 





review_title: Official Blind Review #1

final_score: rating: 6: Weak Accept
review_content: Overview: This paper discusses the risk of membership inference attacks that deep neural networks might face when used in a practical manner on real world datasets. Membership inference attacks can result in privacy breaches, a significant concern for many fields who might stand to benefit from using deep learning in applications. The authors demonstrate how attack accuracy goes up when one dataset is used for training while another altogether is used for testing. They propose the use of causal learning approaches in order to negate risk of membership inference attacks. Causal models can handle distribution shifts across datasets because they learn using a causal structure. 

Contributions: In the theory part of the paper, the authors provide several proofs demonstrating that causal models have stronger differential privacy guarantees than association modes, that causal models trained on large samples are able to protect the dataset against attacks, and that causal models trained on smaller samples still have higher protection than association models trained on similarly sized samples. In addition to theoretical contributions, the authors also provide an experimental evaluation using 4 accepted experimental datasets.


Questions and Comments:
Page 2: “...while association modes exhibit upto 80%...” -> “...up to...”
My expertise is not in causal learning or structures, so I have a few questions about using it in practice. You mentioned that the datasets used in the experimental section were used in order to avoid errors in learning causal structure. 
How likely is it to have these errors using a different dataset? 
How long/how much effort does it take to figure out conditional probability table? Is this a significant amount of time compared to training? Is it automatic or manually done by humans?
If it is done by humans, is it plausible to assume that every dataset implicitly contains a causal structure (not including random walks)?
Your experimental results suggest that the causal model can learn on smaller amounts of data than the DNN. Does this scale for even larger input parameter datasets as well, such as water?
You mention this potentially being used to prevent attacks on real-world applications, such as HIV patient prediction/classification systems. Do you believe that your results prove causal models will scale to datasets that contain these kinds of complex causal structures? 
Could you provide the layer architectures of all three models used for your experiments? Are these out-of-the-box solutions from libraries, or something more custom built?

How would the causal model perform compared to state of the art techniques for these datasets, in both accuracy and attack protection? I understand that isn't the main point of this paper, this is me being curious.

I give this paper a borderline acceptance, based upon the fact that the above questions need to be addressed. I'm not sure its clear to see how the experimental results demonstrate that the causal model definitely outperforms DNNs in all cases. I would like to hear the author's defense of the method when it comes to datasets with higher numbers of features, specifically the water dataset.

review_title: Official Blind Review #3

Meta-review: Maintaining the privacy of membership information contained within the data used to train machine learning models is paramount across many application domains.  Moreover, this risk can be more acute when the model is used to make predictions using out-of-sample data.  This paper applies a causal learning framework to mitigate this problem, motivated by the fact that causal models can be invariant to the training distribution and therefore potentially more resistant to certain privacy attacks.  Both theoretical and empirical results are provided in support of this application of causal modeling.

Overall, during the rebuttal period there was no strong support for this paper, and one reviewer in particular mentioned lingering unresolved yet non-trivial concerns.  For example, to avoid counter-examples raised the reviewer, a deterministic labeling function must be introduced, which trivializes the distribution p(Y|X) and leads to a problematic training and testing scenario from a practical standpoint.  Similarly the theoretical treatment involving Markov blankets was deemed confusing and/or misleading even after careful inspection of all author response details.  At the very least, this suggests that another round of review is required to clarify these issues before publication, and hence the decision to reject at this time.
Decision: Reject