Venue: ICLR 2017 conference

Reviews:

final_score: rating: 9: Top 15% of accepted papers, strong accept
review_content: This paper addresses the problem of achieving differential privacy in a very general scenario where a set of teachers is trained on disjoint subsets of sensitive data and the student performs prediction based on public data labeled by teachers through noisy voting. I found the approach altogether plausible and very clearly explained by the authors. Adding more discussion of the bound (and its tightness) from Theorem 1 itself would be appreciated. A simple idea of adding perturbation error to the counts, known from differentially-private literature, is nicely re-used by the authors and elegantly applied in a much broader (non-convex setting) and practical context than in a number of differentially-private and other related papers. The generality of the approach, clear improvement over predecessors, and clarity of the writing makes the method worth publishing.
review_title: Nice paper, strong accept

final_score: rating: null
review_content: Thanks for the nice paper with inspiring intuitions. The theory analysis is cool, however, the probabilistic bound has quite a number of  empirical parameters, which makes it difficult to understand whether the absolute 100% security is  guaranteed. Thus we have to check the experiments to understand how secure the method is, unfortunately it is not straightforward to evaluate either. 

More specifically, I have two questions related to the experiments.

1) It is not clear from the experiments that how much better the proposed method is compared with the previous works, e.g., [Fredikson et al, 2015] and [Hamm et al, 2016].

2)  I am quite interested in the comparison between GANs vs distillation for semisupervised learning. From the paper my impression is that GAN-based approach is more secure while distillation-based approach may have higher performance. If it is correct, please give more details and analyze why. 

review_title: theory and experiment

final_score: rating: null
review_content: You mentioned future work on tasks in which "the discrete output categories are not as distinctly defined by the salient input space features."

Is this primarily a matter of empirical experimentation, or are there also some technical/theoretical challenges that would need to be addressed? If so, please elaborate.
review_title: What are the challenges involved in your proposed future work?

final_score: rating: 9: Top 15% of accepted papers, strong accept
review_content: Altogether a very good paper, a nice read, and interesting. The work advances the state of the art on differentially-private deep learning, is quite well-written, and relatively thorough.

One caveat is that although the approach is intended to be general, no theoretical guarantees are provided about the learning performance. Privacy-preserving machine learning papers often analyze both the privacy (in the worst case, DP setting) and the learning performance (often under different assumptions). Since the learning performance might depend on the choice of architecture; future experimentation is encouraged, even using the same data sets, with different architectures. If this will not be added, then please justify the choice of architecture used, and/or clarify what can be generalized about the observed learning performance.

Another caveat is that the reported epsilons are not those that can be privately released; the authors note that their technique for doing so would change the resulting epsilon. However this would need to be resolved in order to have a meaningful comparison to the epsilon-delta values reported in related work.

Finally, as has been acknowledged in the paper, the present approach may not work on other natural data types. Experiments on other data sets is strongly encouraged. Also, please cite the data sets used.

Other comments:

Discussion of certain parts of the related work are thorough. However, please add some survey/discussion of the related work on differentially-private semi-supervised learning. For example, in the context of random forests, the following paper also proposed differentially-private semi-supervised learning via a teacher-learner approach (although not denoted as “teacher-learner”). The only time the private labeled data is used is when learning the “primary ensemble.”  A "secondary ensemble" is then learned only from the unlabeled (non-private) data, with pseudo-labels generated by the primary ensemble.

G. Jagannathan, C. Monteleoni, and K. Pillaipakkamnatt: A Semi-Supervised Learning Approach to Differential Privacy. Proc. 2013 IEEE International Conference on Data Mining Workshops, IEEE Workshop on Privacy Aspects of Data Mining (PADM), 2013.

Section C. does a nice comparison of approaches. Please make sure the quantitative results here constitute an apples-to-apples comparison with the GAN results. 

The paper is extremely well-written, for the most part. Some places needing clarification include:
- Last paragraph of 3.1. “all teachers….get the same training data….” This should be rephrased to make it clear that it is not the same w.r.t. all the teachers, but w.r.t. the same teacher on the neighboring database.
- 4.1: The authors state: “The number n of teachers is limited by a trade-off between the classification task’s complexity and the available data.” However, since this tradeoff is not formalized, the statement is imprecise. In particular, if the analysis is done in the i.i.d. setting, the tradeoff would also likely depend on the relation of the target hypothesis to the data distribution.
- Discussion of figure 3 was rather unclear in the text and caption and should be revised for clarity. In the text section, at first the explanation seems to imply that a larger gap is better (as is also indicated in the caption). However later it is stated that the gap stays under 20%. These sentences seem contradictory, which is likely not what was intended.
review_title: A nice contribution to differentially-private deep learning

final_score: rating: 7: Good paper, accept
review_content: This paper discusses how to guarantee privacy for training data. In the proposed approach multiple models trained with disjoint datasets are used as ``teachers'' model, which will train a ``student'' model to predict an output chosen by noisy voting among all of the teachers. 

The theoretical results are nice but also intuitive. Since teachers' result are provided via noisy voting, the student model may not duplicate the teacher's behavior. However, the probabilistic bound has quite a number of  empirical parameters, which makes me difficult to decide whether the security is 100% guaranteed or not.

The experiments on MNIST and SVHN are good. However, as the paper claims, the proposed approach may be mostly useful for sensitive data like medical histories, it will be nice to conduct one or two experiments on such applications. 
review_title: Good theory

final_score: rating: null
review_content: Nice, well-written, and interesting paper. Very general question: what are the most important challenges in the context of privacy preserving specifically in deep learning that you perceive? 
review_title: Clarifications

Meta-review: The paper presents a general teacher-student approach for differentially-private learning in which the student learns to predict a noise vote among a set of teachers. The noise allows the student to be differentially private, whilst maintaining good classification accuracies on MNIST and SVHN. The paper is well-written.
Decision: Accept (Oral)