Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 8: Accept
review_content: Goals
The paper identifies a key challenge in a large class of real world federate learning problems where we also have to ensure user level data privacy. In these settings the modeler can not inspect the raw data samples from the user (due to privacy concerns) and hence all modeling tasks (from data wrangling to hypothesis generation to labeling to model class selection to validation) become far more challenging. The paper proposes that in these circumstances one may use a generative model that learns the data distribution using federated learning methods with provable differentiable privacy guarantees. The generative model can then produce data (unconditional, or conditional on some features or class labels) which can be inspected by the modeler without compromising user privacy. 

Experiments
The authors illustrate the approach using existing federated DP RNN learning methods, and using a slightly novel GAN learning algorithm for images (largely similar to other algorithms). They use these methods to provide two examples: 1) learning a language model from text (word sequences) where there is a bug in pre-processing steps (tokenization); 2) learning a GAN for images of handwriting on checks where there is a pre-processing bug that inverts the grayscale of images. These examples illustrate the potential for such methods to possibly be useful to modelers. While one may quibble about some details (see section below) the experimental set up is reasonable to illustrate the need and some of the challenges modelers are likely to face in the real world (

Evaluation & Questions

I'm really torn because I really enjoyed the paper very much overall but I have some strong concerns as well. 

Positives: the paper is well motivated and very well written (it is really a pleasure to read, and it is very clear about the details -- especially after they release the code it should be possible to reproduce the results too). The authors shine a spot light on a problem that is very important & widespread (eg while learning from condifential data on cell phones). The proposed solution is fairly simple, intuitive, and quite high level (lets use a generative model that creates phantom data that can be inspected)

Negatives: I am not entirely sold on this being a realistic approach in the long term -- ie that some of the key problems will ever be solvable (I'm quite ok even if they are not solved now in the first paperr). The authors do a very good job of being transparent about several potential issues (see eg last paragraphs of main paper and appendix D). My biggest concerns are below:
* the phantom samples generated from the model need to be very realistic in order to be useful. In other words, we need to have excellent, high fidelity generate models. Even to create proper hypothesis, create proper model classes, assess convergence, or assess whether the generative model is good enough one needs to be able to inspect the raw data -- which can not be done in the first place. This can not be entirely automated eliminating need for human inspection -- and the problem is much worse in generative models (which need to encode more information) than in discriminative models which need to encode less information (bits) almost by definition. Thus one has simply traded the problem of needing to inspect data to model the final algorithm (whcih could be discriminative) and has to deal with the problem of needing data to inspect the intermediate, generative model (which is also learned using federates, DP guaranteeing ways). It is not at all clear what one accomplished by doing this. 
* GANs are notoriously hard to train with mode collapse etc.  Setting hyper parameters of any generative model also needs access to original data and impacts the privacy guarantees. 

***NOTE added after author response***
The rebuttal has sufficiently address the quibbles I raised below. I'm leaving it here to allow traceability. I'm not fully convinced about the response to the main issue I raised above (ie if the generative model is not very representative, high-fidelity, then one cant know whether a potential bug discerned by inspecting its samples is an artefact of the generative model or whether it is truly a fundamental bug upstream -- and training a high quality generative model also requires one to inspect the raw data in the first place so the problem has simply been swept under the carpet). Nevertheless for a first paper on the topic I think the contributions and intuition provided here are quite valuable so I am ok leaving this for for future work. 

* quibble#1: theoretical DP bounds are not very tight. For example, in table 2 they may want to use realistic estimates instead of epsilon even to prove their high level point. I'm not sure I can buy their argument even on this illustrative problem as it stands. 
* quibble#2: You may want to at least make an effort to compare against the nearest possible methods in your experimental setup even if they are not a great match to the problem. I'm not intimately familiar with the recent literature but you mention Triascyn & Faltings (2019) so perhaps you could also use that and expand a bit more on the novelty here
review_title: Official Blind Review #1

final_score: rating: 6: Weak Accept
review_content: This work presents a method for using generative models to gain insight into sensitive user data, while maintaining guarantees about the privacy of that data via differential privacy (DP) techniques. This scheme takes place in the federated learning (FL) setting, where the data in question remains on a local device and only aggregate updates are sent to a centralized server. The intended application here is to use the trained generative models as a substitute for direct inspection of user data, thus providing more tools for debugging and troubleshooting deployed models in a privacy conscious manner.

Pros:
Given the growing computational power of mobile devices and the importance of privacy for large-scale deployment of machine learning, this work is a timely contribution that could augment the ML pipeline for at-scale applications dealing with sensitive data. The authors do a good job of fleshing out the intended use cases of their training scheme, and present a pair of experiments that are well-chosen for illustrating the utility of generative models when dealing with private data.

Cons:
Although likely of practical use, the work seems to be lacking in novelty in several respects. First, the techniques developed here represent a fairly straightforward merger of DP and FL tools without much in the way of qualitatively new offerings. While the authors do develop a new GAN training scheme that works in the FL setting, this adaptation is also pretty straightforward, and mostly follows the approach laid out in [1] for training recurrent neural nets.

Secondly, this paper comes in the midst of many other works aiming to integrate different combinations of generative models, privacy, and distributed training (as pointed out in the related work section). While the particular combination of techniques here differ from those in previous work, the authors don't attempt to justify why their training scheme should be preferred over these prior methods. And although their experiments are useful for understanding the general utility of generative models trained in a private and decentralized setting, they unfortunately don't permit any direct comparison with the experiments used in these previous papers.

Verdict:
For the reasons given above, I cannot recommend acceptance of this work.

[1] H Brendan McMahan, Daniel Ramage, Kunal Talwar, and Li Zhang, Learning differentially private recurrent language models, ICLR 2018

*** Follow-up after authors' rebuttal ***

I'd like to thank the authors for their rebuttal, and for the significant addition to the paper in the form of an expanded Section 2. This material has helped me gain a bit better perspective on the use cases for their work, and convinced me of the potential for their methodology within real-world development of deep learning tools and services. In addition, this additional context helps to motivate the two experiments described here as fair representatives of actual debugging problems, and not simply issues that were hand-chosen to prove the authors' point.

I still hold that the paper offers very little in the way of new conceptual or technical contributions, but in light of the potential utility of this privacy-conscious generative pipeline for the broader deep learning community, I have changed my score from a weak reject to a weak accept.
review_title: Official Blind Review #2

final_score: rating: 3: Weak Reject
review_content: This paper proposes a differentially private federated learning method to learn GAN with application to data bugging situations where privacy protection is needed. The proposed method tries to leave the data at the user-end to train the discriminators, and learn the generator at the centralised server. To support the debugging data related issues as claimed, two specific examples related to text and image modeling were presented. It is the generator which is DP-protected (as the discriminators are DP-protected) makes it possible where the generated data can hint the potential bugs. 

The scenario being considered is interesting and two real examples have used to illustrate the idea. However, this paper falls short in the following ways: 
- It adopts what being proposed in McMahan et al. (2018) with some modifications to achieve the goal. The novelty is more related to the proposed application which allows debugging data issues to be possible when the data is private and decentralised.
- The two debugging illustrations are very specific in term of the errors introduced and the ways to achieve the debugging goal. It is not sure how they can be further generalized to other types of bugs.
- The paper is well written. However, the readers should have reasonable background on DP, GAN, federated learning, and generative models, or it will be hard to read through. Having said that, the authors do provide quite comprehensive literature review on related topics. But, then not much space is left for providing the necessary background and details for  the proposed federated learning for GAN with DP (other than referring to Algorithm 1). The experiment section is good.

Specific questions:
- Other than the tokenisation bug and the image insertion bug, can more possible examples be described?
- Can the examples be generalised into some methodologies? And, what are the limits? Will there be data inspection needs which cannot be achieved by this approach? What are they?
review_title: Official Blind Review #3

Meta-review: The paper provides methods for training generative models by combining federated learning techniques with differentiable privacy. The paper also provides two concrete applications for the problem of debugging models. Even though the method in the paper seems to be a standard combination of DP deep learning and federated learning, the paper is well-written and presents interesting use cases.
Decision: Accept (Poster)