Generative Models for Effective ML on Private, Decentralized Datasets

1 INTRODUCTION . Real-world systems increasingly depend on machine learning ( ML ) to make decisions , detect anomalies , and power products . Applications ranging from fraud detection to mobile phone keyboards use models trained on data that may be privacy sensitive . The data may contain financial , medical , or behavioral information about individuals . Institutions responsible for these applications of ML must balance data stewardship obligations—including minimizing the risks of data loss , theft , or abuse—with the practical needs of the “ modeler ” whose job is to develop and improve the machine learned models . Modelers working with privacy-sensitive data face significant challenges in model development and debugging . Often , a modeler ’ s first step would be to inspect individual examples in order to discover bugs , generate hypotheses , improve labeling , or similar . But direct inspection of data may be audited or disallowed by policy in some settings . In other settings—including federated learning ( FL ) ( McMahan & Ramage , 2017 ; McMahan et al. , 2017 ) , which is the motivation and focus of this work—the data can not be inspected . In FL , raw data remains distributed across a fleet of devices , such as mobile phones , while an orchestrating server coordinates training of a shared global model . Only the final model parameters and statistics are gathered and made available to the modeler . 1 How can a modeler effectively debug when training data is privacy sensitive or decentralized ? This paper demonstrates that the novel application of auxiliary models , namely privacy-preserving generative models , can stand in for direct data examination during the process of debugging data errors during inference or training . By combining ideas from deep generative models , FL , and user-level differential privacy ( DP ) , we show how some needs traditionally met with data inspection can instead be met by generating synthetic examples from a privacy-preserving federated generative model . These examples could be representative of all or a subset of the non-inspectable data , while at the same time preserving the privacy of individuals . Our contributions include : • Identifying key challenges in implementing end-to-end workflows with non-inspectable data , e.g. , for debugging a ‘ primary ’ ML model used in a mobile application . • Proposing a methodology that allows ( sufficiently powerful ) ‘ auxiliary ’ generative models to resolve these challenges . • Demonstrating how privacy preserving federated generative models—RNNs for text and GANs for images—can be trained to high enough fidelity to discover introduced data errors matching those encountered in real world scenarios . This requires a novel adaption of generative adversarial networks ( GANs ) to the federated setting with user-level DP guarantees . 2 CHALLENGES OF ML ON NON-INSPECTABLE DATA . A key aspect of deep learning is the utilization of large datasets . The behavior of a deep network is tightly coupled to the data it trains on . As Sculley et al . ( 2015 ) puts it , “ ML is required in exactly those cases when the desired behavior can not be effectively expressed in software logic without dependency on external data. ” The centrality of data to ML performance is recognized in the attention paid to data analysis , curation , and debugging . Textbooks devote chapters to methodologies for “ Determining Whether to Gather More Data ” ( Goodfellow et al. , 2016 ) . The literature conveys practical lessons learned on ML system anti-patterns that increase the chance of data processing bugs ( Sculley et al. , 2015 ) . In general , an assumption of unfettered access to training or inference data is made ( Riley , 2016 ; Zinkevich , 2017 ; Chakarov et al. , 2016 ) . What to do if direct access is precluded , e.g. , if working with private and decentralized data , is largely unaddressed . Below , we describe six common tasks ( T1–T6 , summarized in Table 1 ) where a modeler would typically use direct data access . Our choice of these tasks is validated by Humbatova et al . ( 2019 ) , a recent survey providing a taxonomy of faults in deep learning systems . Two of the largest classes of faults are in ‘ Preprocessing of Training Data ’ and ‘ Training Data Quality ’ ; T1–T6 are manners of addressing these faults . Sanity checking and model debugging ( T1–T4 ) Experienced modelers will often inspect some random examples and observe their properties before training a model ( T1 ) ; or if this step is skipped , it is often a first step in debugging . Are the size , data types , and value ranges as expected ? For text data , are words or word pieces being properly separated and tokenized ? A modeler is often working to identify mistakes in upstream processing , affecting all or most data , and readily apparent by looking at one or a few input samples . When a model is misbehaving , looking at a particular subset of the input data is natural . For example , in a classification task , a modeler might inspect misclassified examples to look for issues in the features or 1Despite these limitations , FL systems ( Bonawitz et al. , 2019 ) have proven effective for training and deploying real-world models , for example see Hard et al . ( 2018 ) ; Yang et al . ( 2018 ) ; Chen et al . ( 2019 ) . labels ( T2 ) . For tasks where the full set of possible labels is too large , e.g . a language model with a fixed vocabulary , the modeler might examine a sample of out-of-vocabulary words ( T3 ) . For production ML applications , it is often important to monitor not only overall accuracy , but accuracy over finer grained slices of the data — for example , by country , by class label , by time of day , etc.2 If low accuracy is observed on such a slice , it is natural to examine examples selected from that segment of the data ( T4 ) . For example , if training on data which can be grouped by user , it is natural ( but potentially problematic from a privacy perspective ) to look at data from users with low overall accuracy . Data labeling ( T5 ) Supervised learning problems require labeled data . Often in FL , labels can be inferred naturally from context on device—for example , the next-word predictor in a keyboard app ( e.g. , as in Hard et al . ( 2018 ) ) is a problem where the input is a sequence of words and the label is the subsequent word . But some supervised learning problems are better suited to human raters inspecting and manually assigning class labels ( e.g. , visual object recognition datasets as in Deng et al . ( 2009 ) ) . Because FL systems do not allow users ’ data to be sent to the cloud for labeling , it would be desirable to synthesize realistic , representative examples from the decentralized data that could be labeled by human raters . Doing so would require a private mechanism of generating high-fidelity training examples . This would expand the set of ML problems that can be afforded FL ’ s privacy advantages . Detecting bias in training data ( T6 ) Another common use of human raters in supervised learning is to assign labels to non-privacy-sensitive training examples curated from a public dataset or collected via donation . These examples might be the only ones a classifier is trained on , even if a much larger unlabeled dataset exists ( e.g . on phones in FL ) . The distributional difference between the datasets is a source of bias . For example , a labeled training dataset for a face detector might not be representative of the diversity of racial/ethnic/gender/age classes that the face detector sees through a phone ’ s camera ( i.e. , at serving time ) . These classes can be unrelated to the model ’ s task but serve as a confounding factor in prediction if the model has erroneously fit an assumption about them during training ( e.g. , a smiling face detector that judges smiling vs. non-smiling based on gender or gender expression ( Denton et al. , 2019 ) ) . When entire groups are unor underrepresented in training , the result can be biased predictions that induce quite insidious social effects ( Angwin et al. , 2016 ) . Modelers ’ goals of building unbiased classifiers could be met by inspecting examples of classes missing at training but present at prediction time . These examples—even at low fidelity—could indicate whether a model is fair and point to where additional training data collection is required . 2E.g. , McMahan et al . ( 2013 , Sec . 5.2 ) notes 100s of slices of relevance for ad-click-through-rate prediction . 2.1 USING GENERATIVE MODELS INSTEAD OF DATA INSPECTION . The general methodology for using generative models in place of data inspection is as follows : in a situation where the modeler would otherwise inspect examples based on a particular criteria ( as shown in Table 1 ) , this criteria is expressed as a programmatic data selection procedure used to construct a training dataset for a generative model . In the case of FL specifically , this might involve both selecting only a subset of devices to train the model , as well as filtering the local dataset held on each device . We conduct experiments for three such tasks in this paper . Section 5 considers T3 , and shows how the methodology proposed here can be extended when the primary model is itself a generative model . We conduct experiments for T2 and T4 in Section 6 , where both approaches provide alternative ways of discovering the same bug ; results for the T4 approach are presented in the main body , with T2 results appearing in the Appendix . We chose these scenarios as being representative of a large class of data-related ML bugs , based on real-world experience in both industry and academic settings . Independently , Humbatova et al . ( 2019 ) identifies ‘ text segmentation ’ and ‘ pixel encoding ’ as being very common sources of bugs . By showing that privacy-preserving federated generative models can debug these scenarios , we demonstrate the applicability of the approach . One of the reasons we select these scenarios is that high-fidelity generation is not necessary to detect these bugs , and pushing the state-of-the-art in generative models is not the aim of this work . In contrast , for some classes of bugs , as well for generating examples used for labeling and training ( T5 ) , high-fidelity private generative models are necessary . We hope that this work will serve as a call-to-action for the generative modeling community , as our work strongly suggests that these models have a fundamental role to play in enabling the widespread use of privacy-preserving ML workflows . Task T6 requires a more subtle approach in defining the selection criteria . We briefly describe possible approaches in the spirit of providing a more complete description of the uses of generative models in private ML workflows , but leave details and experiments as promising directions for future work . Suppose we have a public , centralized distribution Ppublic as well as a private , decentralized distribution Pprivate . Informally , we might wish to generate examples according to x ∼ Pprivate ( x ) Ppublic ( x ) or perhaps x ∼ Pprivate ( x | Ppublic ( x ) ≤ ) . If we have a family of generative models for the domain of x that explicitly provide a likelihood ( e.g. , the language models we consider in Section 5 ) , then we can first train such a generative model on Ppublic , and use this to filter or re-weight the decentralized samples drawn from Pprivate to train a second generative model which will be used to generate samples representative of Pprivate but not Ppublic . In the case of architectures that do not provide an explicit likelihood ( e.g. , GANs ) , alternative techiques are needed , for example a secondary discriminator used to differentiate samples from Pprivate from those from Ppublic . Of course , if any of the generative models used here simply reproduce the private examples of users , nothing has been gained . Hence , we next turn to training these models with strong privacy guarantees .

Summary: This work presents a method for using generative models to gain insight into sensitive user data, while maintaining guarantees about the privacy of that data via differential privacy (DP) techniques. This scheme takes place in the federated learning (FL) setting, where the data in question remains on a local device and only aggregate updates are sent to a centralized server. The intended application here is to use the trained generative models as a substitute for direct inspection of user data, thus providing more tools for debugging and troubleshooting deployed models in a privacy conscious manner.