An Investigation towards Differentially Private Sequence Tagging in a Federated Framework (2021)

Abstract: To build machine learning-based applications for sensitive domains like medical, legal, etc. where the digitized text contains private information, anonymization of text is required for preserving privacy. Sequence tagging, e.g. as done in Named Entity Recognition (NER) can help to detect private information. However, to train sequence tagging models, a sufficient amount of labeled data are required but for privacy-sensitive domains, such labeled data also can not be shared directly. In this paper, we investigate the applicability of a privacy-preserving framework for sequence tagging tasks, specifically NER. Hence, we analyze a framework for the NER task, which incorporates two levels of privacy protection. Firstly, we deploy a federated learning (FL) framework where the labeled data are not shared with the centralized server as well as the peer clients. Secondly, we apply differential privacy (DP) while the models are being trained in each client instance. While both privacy measures are suitable for privacy-aware models, their combination results in unstable models. To our knowledge, this is the first study of its kind on privacy-aware sequence tagging models.

## introduction the emergence of substantial amounts of digitized unstructured text gives rise to train machine learning models for various downstream applications. but for sensitive domains like medical, legal, etc., the text documents contain private sensitive information, which is supposed to be anonymized for preserving privacy. the first step for anonymizing text is to detect the span of the private information and identify the type of information. sequence tagging is the kind of task that can help in anonymization. however, to train such a sequence tagging model, a significant amount of labeled data are required. but the labeled data contains -by definition -private sensitive information and are often divided across different data silos. there are legal regulatory policies as well by the us health insurance portability and accountability act (hipaa) foot_0 and eu general data protection regulation (gdpr) (agencia-espanola-proteccion-datos, 2019), which restricts such sensitive data access. on the other hand, to train a centralized machine learning model, there is a requirement of aggregating such distributed data in a central server, which can cause privacy breaches. hence, there is a requirement for a privacy-preserving sequence tagging framework that complies with the data protection policies. federated learning (fl) @xcite ) is one such paradigm that provides a framework to train a centralized machine learning model without sharing the distributed data from different data silos. since the raw data from different data silos are not being shared in this framework, the primary level of privacy is maintained. however, the fl framework is also vulnerable to several inference attacks @xcite @xcite in some scenarios. to mitigate such inference attacks, differential privacy @xcite was developed, which comes with a theoretically guaranteed measurement of privacy. there have been many recent research works on deploying the fl framework in several applications like image classification @xcite , emotion detection @xcite , anonymization @xcite , robotics (imteaj and amini, 2020), etc. and also in medical domain @xcite @xcite . researchers also investigated the scope of the differentially private algorithm in several applications @xcite @xcite . however, for sequence tagging tasks, the applicability of the fl framework along with differential privacy (dp) is yet to be explored. for our study, we focus on one such sequence tagging task, namely named entity recognition (ner). we argue that this is quite close to a sequence-tagging based anonymization task and it allows us to study the effects of privatization measures in the task performance. in this paper, we prepare an fl framework following the federatedaveraging @xcite approach and deploy a differentially private stochastic gradient descent @xcite optimizer while training to obtain differential privacy for the ner task. as base ner models, we consider two variants of one of the state-of-the-art approaches for sequence labeling, that uses the lstm variant @xcite of bidirectional recurrent neural networks (bilstms). note that the aim of our work is not to produce a state-of-the-art performance for ner tasks. rather, our focus is to analyze how the performance of ner models varies in a differentially private federated learning framework. therefore, we do a comprehensive analysis using conll 2003 english ner dataset and investigate the effect of different levels of privacy on the performance of ner models. ## methodology as we deal with the named entity recognition (ner) task, we attempt with two variations of bilstm based models. in one variant we keep timedistributed dense layer (tddl) as the final layer with activation function -'softmax' and in another variant, we keep the conditional random field (crf) @xcite as the final layer. in a nutshell, both the variants have three layers: input embedding layer, bidirectional lstm layer, and either tddl or crf as the final layer. we refer to those model variants 'bi-lstm-tddl' and 'bi-lstm-crf' respectively for our study. next, we will discuss the two privacy mechanisms: federated learning and differential privacy. federated learning: the objective of the federated learning framework is to train a centralized model from data distributed across multiple client data silos, eliminating the need for raw data sharing. we adapt the well-accepted federatedaveraging algorithm proposed by @xcite . as per the process, first, a global model from the server site is shared across n client sites. next, each client site trains the model based on its local data. after the training is complete, the parameter updates of the local models are then subsequently sent to the central aggregation server. next, the central server computes the average of all the model parameters over n clients and updates the global model accordingly. the whole process continues for specified number of rounds (t) and in each round a random set of m clients ( @xmath0. for this study, in each round we allow all the n client sites to participate in the process. ## experiments and discussions since the objective of our work is to investigate the effect of federated learning (fl) framework and differential privacy (dp) on the state-of-the-art ner model, we design our analysis in four different phases. first, we observe the basic ner model performance without using fl as well as dp. in the second phase, we incorporate dp in the optimizer of the ner model and analyze its behavior. third, we analyze the performance of the ner model in an fl framework, but none of the optimizers of the clients are dp enabled. in the fourth phase, we investigate the performance of the ner model by deploying the fl framework and incorporating dp into the client-side optimizers. for our analysis, we use conll 2003 english ner dataset, having a training set size of 14987, validation set size of 3466, and test set size of 3684. each token in the dataset is tagged with other (o) or one of the four entity types: person (per), location (loc), organization (org), miscellaneous (misc). considering the bio2 annotation standard, the number of possible labels for each token is 9. without dp, with fl: next, we analyze the fl setup for ner, where we have one centralized server and n number of the client sites. the training and validation data are divided equally among all the clients. we analyze the framework in two ways. first, we observe the performance variation of the aggregated model on test data with the number of clients, which is presented in ## conclusion in this work, we presented an analysis of the behavior of one sequence tagging task namely ner in a federated learning framework along with differential privacy, which is the first-ever attempt of its kind. from the investigation, we observed that with dp-sgd optimizer, the performance of crf based model tends to decrease significantly in our current experimental setup. in the federated framework, we observed that with the increase of the number of clients with smaller training data, the performance of aggregated models decrease significantly, whereas the performance shoots up when client models are trained in an incremental approach and the incremental models are communicated to the server after each training phase. on the other hand, when a dp-sgd optimizer is deployed in each client training phase, even the incremental training policy works only till 60% of the training data is used and thereafter performance drops. immediate future work would be to find out the root cause of such a phenomenon and mitigate it, as a combination of privacy measures would be very much desired in privacy-aware application scenarios. to extend this study, we plan to explore several other ner datasets (even low-resource datasets) to find out more general behavior. hyper-parameter tuning is another direction of our future work to find out the best performance for each set-up. the broader goal is to build a differentially private federated framework for sequence tagging tasks with compromising performance as little as possible.