User-generated textual data is rich in content and has been used in many user behavioral modeling tasks . However , it could also leak user private-attribute information that they may not want to disclose such as age and location . User 's privacy concerns mandate data publishers to protect privacy . One effective way is to anonymize the textual data . In this paper , we study the problem of textual data anonymization and propose a novel Reinforcement Learning-based Text Anonymizor , RLTA , which addresses the problem of private-attribute leakage while preserving the utility of textual data . Our approach first extracts a latent representation of the original text w.r.t . a given task , then leverages deep reinforcement learning to automatically learn an optimal strategy for manipulating text representations w.r.t . the received privacy and utility feedback . Experiments show the effectiveness of this approach in terms of preserving both privacy and utility . Social media users generate a tremendous amount of data such as profile information , network connections and online reviews and posts . Online vendors use this data to understand users preferences and further predict their future needs . However , user-generated data is rich in content and malicious attackers can infer users ' sensitive information . AOL search data leak in 2006 is an example of privacy breaches which results in users re-identification according to the published AOL search logs and queries ( Pass et al . , 2006 ) . Therefore , these privacy concerns mandate that data be anonymized before publishing . Recent research has shown that textual data alone may contain sufficient information about users ' private-attributes that they do not want to disclose such as age , gender , location , political views and sexual orienta-tion ( Mukherjee and Liu , 2010 ; Volkova et al . , 2015 ) . Little attention has been paid to protect users textual information ( Li et al . , 2018 ; Zhang et al . , 2018 ; Anandan et al . , 2012 ; Saygin et al . , 2006 ) . Anonymizing textual information comes at the cost of losing utility of data for future applications . Some existing work shows the degraded quality of textual information ( Anandan et al . , 2012 ; Zhang et al . , 2018 ; Saygin et al . , 2006 ) . Another related problem setting is when the latent representation of the user generated texts is shared for different tasks . It is very common to use recurrent neural networks to create a representation of user generated text to use for different machine learning tasks . Hitaj el al . show text representations can leak users ' private information such as location ( Hitaj et al . , 2017 ) . This work aims to anonymize users ' textual information against private-attribute inference attacks . Adversarial learning is the state-of-the-art approach for creating a privacy preserving text embedding ( Li et al . , 2018 ; Coavoux et al . , 2018 ) . In these methods , a model is trained to create a text embedding , but we can not control the privacyutility balance . Recent success of reinforcement learning ( RL ) ( Paulus et al . , 2017 ; Sun and Zhang , 2018 ) shows a feasible alternative : by leveraging reinforcement learning , we can include feedback of attackers and utility in a reward function that allows for the control of the privacy-utility balance . Furthermore , an RL agent can perturb parts of an embedded text for preserving both utility and privacy , instead of retraining an embedding as in adversarial learning . Therefore , we propose a novel Reinforcement Learning-based Text Anonymizer , namely , RLTA , composed of two main components : 1 ) an attention based task-aware text representation learner to extract latent embedding representation of the original text 's content w.r.t . a given task , and 2 ) a deep reinforcement learning based privacy and utility preserver to convert the problem of text anonymization to a one-player game in which the agent 's goal is to learn the optimal strategy for text embedding manipulation to satisfy both privacy and utility . The Deep Q-Learning algorithm is then used to train the agent capable of changing the text embedding w.r.t . the received feedback from the privacy and utility subcomponents . We investigate the following challenges : 1 ) How could we extract the textual embedding w.r.t . a given task ? 2 ) How could we perturb the extracted text embedding to ensure that user privateattribute information is obscured ? and 3 ) How could we preserve the utility of text embedding during anonymization ? Our main contributions are : ( 1 ) we study the problem of text anonymization by learning a reinforced task-aware text anonymizer , ( 2 ) we corporate a data-utility taskaware checker to ensure that the utility of textual embeddings is preserved w.r.t . a given task , and ( 3 ) we conduct experiments on real-world data to demonstrate the effectiveness of RLTA in an important natural language processing task . In this paper , we propose a deep reinforcement learning based text anonymization , RLTA , which creates a text embedding such that does not leak user 's private-attribute information while preserving its utility w.r.t . a given task . RLTA has two main components : ( 1 ) an attention based taskaware text representation learner , and ( 2 ) a deep RL based privacy and utility preserver . Our results illustrate the effectiveness of RLTA in preserving privacy and utility . One future direction is to generate privacy preserving text rather than embeddings . We also adopt deep Q-learning to train the agent . A future direction is to apply different RL algorithms and investigate how it impacts results . It would be also interesting to adopt RLTA for other types of data .

Challenge: User-generated texts in social media can leak sensitive private information about users however methods for protection have not been investigated well.
Approach: They propose a reinforcement leaning-based text analyzer which prevents private-attribute leakage while keeping the utility by extracting representations and manipulating with privacy and utility feedback.
Outcome: The proposed method effectively preserves privacy and utility in textual embeddings in experiments with real-world data.