Identifying and Classifying Third-party Entities in Natural Language Privacy Policies (2020)

Abstract: App developers often raise revenue by contracting with third party ad networks, which serve targeted ads to end-users. To this end, a free app may collect data about its users and share it with advertising companies for targeting purposes. Regulations such as General Data Protection Regulation (GDPR) require transparency with respect to the recipients (or categories of recipients) of user data. These regulations call for app developers to have privacy policies that disclose those third party recipients of user data. Privacy policies provide users transparency into what data an app will access, collect, shared, and retain. Given the size of app marketplaces, verifying compliance with such regulations is a tedious task. This paper aims to develop an automated approach to extract and categorize third party data recipients (i.e., entities) declared in privacy policies. We analyze 100 privacy policies associated with most downloaded apps in the Google Play Store. We crowdsource the collection and annotation of app privacy policies to establish the ground truth with respect to third party entities. From this, we train various models to extract third party entities automatically. Our best model achieves average F1 score of 66% when compared to crowdsourced annotations.

## introduction according to statistics from google play store 1 and app store 2 there are about 2.2 and 2.8 million applications (apps) in various categories, available 1 play.google.com 2 https://www.apple.com/ios/app-store/ for download on each platform, respectively. to generate revenue from targeted advertising, apps often collect user data such as real-time location, financial information, friends list, photos, and contact information, among others. apps inform users of this activity through privacy policies, which detail the kinds of personal data being collected, how the data is being used, and with whom the data is shared. consumer surveys reveal that over 90% of users accept legal terms and conditions without reading them @xcite . moreover, statistics from a younger group with ages 18 to 34, showed that more than 97% of users never read the privacy policy and were unaware of the fact that their data has been shared to various third party companies @xcite . such challenges arise due to the following facts. first, privacy policies are too complex and long-winded @xcite , and consumers lack the bandwidth to read and comprehend each and every privacy policy for all the services they use. second, the asymmetric power between businesses and consumers yields "take-it-or-leave-it" attitude forced by businesses @xcite , where a consumer who doesn't like the terms being offered, is likely to be dismissed because the loss will be negligible. recent incidents highlight the importance of user awareness concerning third-party data collection in mobile apps. in august 2020, following a lawsuit brought about by the los angeles city attorney's office, the weather channel agreed to give users control over how their location data would be shared 3 . the lawsuit alleged the weather channel app improperly shared location data with third parties, when consumers only con-sented to using it to find local weather reports. similarly, the us federal trade commission settled with children's app developer hyperbeard inc. for allowing advertisers to collect tracking information from young users, in violation of the children's online privacy protection act 4 . recent regulatory frameworks, such as general data protection regulation (gdpr) and california consumer privacy act (ccpa), require app development companies to be more concrete on the data recipients listed in the privacy policies and disclose the categories of third parties with whom the app may share personally identifiable data. such third parties, include affiliates, advertising networks, and analytics providers. previous works have proposed machine learning models to extract categories of data practices, such as data collection, usage, and sharing statements from privacy policies @xcite ; @xcite . these models utilize natural language processing techniques to identify relevant features for automated extraction. however, these models fail to identify and categorize third party entities in sharing data practices of mobile apps. in this paper, we aim to analyze sharing data practices in privacy policies of mobile apps to develop an automated approach to extract and categorize third party entities. the third party entities can fall into two main categories: (1) generic third party entities, such as advertisers, partners, and affiliates; and (2) specific third party entities, such as google analytics, facebook, and paypal. for this reason, we first collect privacy policies of 100 mobile apps from spain, which were associated with 485 most downloaded apps in google play. we use crowdsourcing to annotate and extract the third party categories and names from these policies. these annotations are used to train and evaluate implementations of stanford corenlp crf, scikit-learn crf, bi-lstm, and bi-lstm crf as named entity extraction (ner) models. through this research, we study the feasibility of automatically extracting third party entities from privacy policy text using the crowdsourced annotated data. to summarize, the contributions of this paper are two fold: (1) empirical evaluation of a pretrained model to extract third party entities from privacy policies; (2) training and evaluation of var-ious models to extract and categorize third party entities using the annotated data from 100 privacy policies. our results indicate poor performance of the pre-trained ner model on privacy policy corpus and the importance of training ner models for domain-specific purposes. further, our experiment results suggest that natural language features in stanford corenlp crf model provides the best performance for extracting and categorizing third party entities. this model achieves f1 scores of 79% and 53% for generic and specific third party extraction, respectively. this paper is organized as follows. in § 2 and § 3, we discuss background and related work. in § 4 we introduce our method in text analysis of privacy policies. in § 5, we present the evaluation and results, followed by discussion, threats to validity, and concluding remarks in § 6, § 7, and § 8. ## related work researchers have proposed methods to ensure data transparency and compliance by analyzing data practices expressed in privacy policies. for example, breaux et al. formalized data practice requirements from privacy policies using description logic to automatically detect conflicting requirements and to trace data flows across policies of interacting services @xcite . this method relies on a small set of manually annotated privacy policies to instantiate the language. further, initial attempts in automated analysis of privacy policy text have largely focused on readability scores @xcite ; @xcite ; @xcite . prior to introduction of gdpr, studies have focused on identifying policy sections related to different data practices (e.g., collection, use, and sharing) using machine learning models trained on web-based apps' privacy policies @xcite al. (2014); liu et al. (2018); wilson et al. (2016); zimmeck et al. (2017) ; @xcite . in these studies, each segment of the privacy policy is mapped to data types from a pre-defined set of keywords, including "location", "contact", or "identifier." these studies fail to automatically extract the exact data types from the segments and identify data types that don't exist in the pre-defined set. further, approaches that map data types to a small number of broad categories introduce inaccuracies because the mapping is based on satisficing where subtle differences between the meanings of two or more types are ignored ?. for example, the phrase "wifi ssid" can be construed to mean a type of location information @xcite when location is one of a few categories to choose from (i.e., the categorization fits given the constraints). however, the term actually describes a technology that must be combined with the ssid's device information (e.g., mac address) before it can be indirectly correlated with location; it should not be directly related to location on its own. gdpr requires the data controllers to disclose, among others, the data recipients' categories or names (while the wp29 suggests that "the default position is that a data controller should provide information on the actual (named) recipients of the personal data"). since 2018, majority of the privacy polices introduced updates in their terms and changes to their structure to be compliant with the new regulations. therefore, we focus on creation of a corpus containing recently published privacy policies for our analysis. @xcite . researchers also utilized string searching techniques to find third-party domain names in privacy policy text @xcite . in contrast to these studies, our goal is to automatically extract any mentions of third party entities, including generic third party names (e.g., third parties, advertisers, affiliates, analytics services) and specific third party entities (e.g., google analytics, amazon web services, facebook, paypal) from mobile apps privacy policies. ## privacy policy analysis in this section, we first discuss our approach for constructing a corpus of 100 privacy policies. this corpus is used to train and evaluate various models to extract and classify third party entities. second, we introduce a pilot study on five privacy policies selected from the corpus. through the pilot study, we provide evidence on why pre-trained named entity recognition (ner) models on news corpus, such as stanford corenlp crf foot_3 cannot provide desirable results. third, we describe our crowdsourcing approach to annotate the privacy policy corpus. finally, we utilize the annotations toward training and evaluation of various ner models. ## evaluation and results we view our work as an exploration of extracting and classifying third party entities from privacy policies. as mentioned in § 4.4, we implement six ner models that utilize natural language (nl) features, bag-of-words (bow), and word2vec (w2v). using the train and test split of 100 privacy policies in our corpus, we train and evaluate each model. each token in the annotated corpus is tagged with one of three labels: generic, specific, and none. the criteria for this selection is described in § 4.4. ## discussion we start our discussion by analyzing results from our pilot study. through this study, we compared the experts annotations with automated third party entities extracted using a pre-trained crf model. however, low precision, recall, and f1 score suggest poor performance of this pre-trained model for our task. the crf model used in our pilot study was trained on news wire corpus. the tags in this pre-trained model also does not satisfy our ner task requirements. using the pre-defined tags, we are only able to classify third party entities in privacy policies using a label called organizations (org) in the crf model implementation. therefore, for calculating performance measures in pilot study, we defined true positives (tps) as an automatically extracted entity that is annotated as either generic or specific by our experts. through further analysis, the pre-trained crf model can only extract eight unique generic third party entities, and 18 unique specific third party entities. the results from our pilot study justifies the need for training ner models using privacy policy corpus. therefore, through this study, we construct an annotated privacy policy corpus that contains third party entities tagged with generic and specific labels. the annotations is the result of hiring qualified workers (i.e., required to successfully pass a qualification test) on amt. finally, the annotated corpus is used to train and test ner models. our experiment results suggest that the natural language (nl) features used in crf implementation of stanford corenlp produces the best results when compared to our gold standard in the test set. this model considers combinations of nl features, including current word, previous word, next word, word character ngrams (up to length six), current and surrounding pos tag sequences, additional word shape features, and presence of word in left an right windows (size 4) 9 finkel et al. (2005). finally, we believe the results can be improved by increasing the size of our annotated corpus. to support this claim, we present the distribution of tagged tokens in our annotated corpus in ## threats to validity construct validity construct validity reflects whether the measurements actually measure that which they are intended to measure yin (2017). in this paper, we provide definitions for "generic third party" and "specific third party." these definitions are reviewed by a legal expert on our team. ## conclusion and future direction in this paper, we aim to analyze sharing data practices of mobile apps and automatically extract and classify third party entities. for this reason, we define two categories for third party entities: (1) generic third parties, such as advertisers, partners, and affiliates; and (2) specific third party entities, including google analytics, facebook, and paypal. we first evaluate the performance of a pre-trained named entity recognition (ner) model on a set five of annotated privacy polices. the poor results of this pilot study suggest that pre-trained ner models on newswire corpus cannot be applied to extract third party entities from privacy polices. therefore, we construct an annotated privacy policy corpus and utilize this corpus to train three different ner models. the annotated privacy policy corpus is constructed through crowdsourcing. finally, our results shows an improvement over pilot study. further, stanford corenlp that implements crf model using various combinations of natural language features associated with words in our corpus results in best performance when compared to the annotations. for future, we foresee to downsample the segments of privacy polices that are not annotated as third parties. using this strategy, we believe training and testing the stanford corenlp crf model will result in better performance. further, regulators can utilize the results from this work to analyze the compliance of privacy policies in scale. the results of this work can also be applied to verify data practices of mobile apps. therefore, the mobile app ecosystem can ensure a transparent environment for users.