AWARE - TEXT : An Android Package for Mobile Phone Based Text Collection and On-Device Processing (2023)

Abstract: We present the AWARE-text package, an open-source software package for collecting textual data on Android mobile devices. This package allows for collecting short message service (SMS or text messages) and character-level keystrokes. In addition to collecting this raw data, AWARE-text is designed for on device lexicon processing, which allows one to collect standard textual-based measures (e.g., sentiment, emotions, and topics) without collecting the underlying raw textual data. This is especially important in the case of mobile phones, which can contain sensitive and identifying information. Thus, the AWARE-text package allows for privacy protection while simultaneously collecting textual information at multiple levels of granularity: person (lifetime history of SMS), conversation (both sides of SMS conversations and group chats), message (single SMS), and character (individual keystrokes entered across applications). Finally, the unique processing environment of mobile devices opens up several methodological and privacy issues, which we discuss.

## introduction unlike traditional nlp tasks (e.g., machine translation or question answering), nlp in the context of psychological, social, and health sciences is aimed at understanding how textual data can characterize people. this includes stance or sarcasm at document-level @xcite , state-level tasks, such as emotion prediction @xcite , trait-level tasks, such as personality prediction @xcite or mental health applications @xcite , or even population-level tasks, for example, monitoring the opioid epidemic via social media data @xcite . similarly, keystroke data (or typing dynamics, i.e., a succession of individual depressions on a keyboard) has been used to predict emotions @xcite and cognition @xcite . historically, human-generated textual data for such social science-oriented tasks is collected from social media (e.g., facebook, reddit, or twitter), open-ended survey questions @xcite , or interviews @xcite . however, more recently, short message service (sms or text messaging) has received attention as a viable data source @xcite @xcite @xcite @xcite . sms, and, more generally, mobile phone-based data, is important for just-in-time adaptive interventions (jitais), which can be used to deliver personalized support and interventions in response to a person's changing physical and mental health @xcite . aware-text 1 is an android mobile phone application (or "app") built to collect passive mobile data (e.g., gps locations and accelerometer data) with particular emphasis on textual 2 data such as sms and keystrokes. this app allows researchers to collect raw textual data, both historical data and prospective data in real time, as well as lexical-based measures calculated on the device. on-device processing of lexical-based measures, such as sentiment or topics, inherently preserves privacy: summary scores (e.g., sentiment) are transmitted to a remote server and the underlying raw data, which can be highly sensitive and contain personally identifiable information (pii), does not need to leave the mobile device. in summary, aware-text focuses on two types of textual data (sms data and keyboard input) across four levels of granularity (person, conversation, message, and character). across all data types and levels aware-text offers the ability to collect raw data as well as lexical measures, which inherently preserve privacy, as shown in figure 2 . this package offers researchers the ability to collect fine-grained textual data which can be used to gain insight into tasks across natural language processing, psychology, computational social sciences, and psycho-linguistics. ## overall system design the aware-text package is an extension of the aware mobile sensing framework, an opensource package developed to passively collect mobile phone sensor data, such as accelerometer, gyroscope, and gps data @xcite . this extension consists of two on-device plugins (to collect sms data and process lexical measures) and a series of post-processing scripts for data aggregation and cleaning. thus, aware-text is able to collect everything aware is able to collect, plus additional textual data. while the aware framework is available for both ios and android devices, the aware-text package is only available on android devices due to ios restrictions on access to raw sms and keyboard data. the high-level features of aware-text are shown in figure 1 . here we see that aware-text pulls data from both the mobile device's keyboard and local sms database (described below). both data types are then optionally processed into lexical measures, after which the raw and processed lexical data are stored in a temporary local database. this data is transferred (via a secure https connection) to a remote server and stored in a final database. aware-text is designed to when referring to data in text form. optimally transfer data whenever wifi connections are available to minimize the amount of cellular data the application uses. the temporary database is then cleared so as not to duplicate data. ## data types aware-text collects two types of raw text data: sms and keystrokes. while the keystroke data is available in the original aware implementation, the keystrokes lexical processing is novel to aware-text, as is the sms collection. ## levels of data collection aware-text has been designed to enable analysis of textual data at various levels of granularity: person, conversation, messages, and characters. this is true of both raw data and lexical measures. this flexibility is enabled by collecting data at a low level and preserving summary statistics that may be aggregated to higher levels of analysis. person-level data is available by aggregating raw text (sms or keystrokes) or lexical measures across a person's individual inputs (for example, their lifetime sms history). conversations can be constructed by combining sms data between pairs and groups of people. this can include sms from nonconsenting individuals (see ethical considerations below). message data (raw or lexical) is obtained by single sms or complete keyboard input and is the most basic unit available for lexical measures. finally, raw character-level data is obtained through keystroke inputs. ## privacy preserving lexical measures running lexical measures on the device allows researchers the ability to collect data across each sms or keystroke without necessitating the collection of the underlying raw data. this raw data could include sensitive information (e.g., revealing search histories or sms with pii) and data from non-consenting individuals (e.g., sms from people communicating with the study participant). as shown in figure 2 , the lexical measures can be applied to both ends of the conversation, thus obfuscating both the exact people in the conversation and their exact utterances while still preserving both the overall conversation and individual turns within the conversation. while mobile phones can collect data in multiple languages, this tokenizer is designed for english text, and thus would need to be replaced for on-device processing of non-english languages. preprocessing applications on the android os are java and, more recently, kotlin based. therefore, we use a java-based tokenizer from the natural language processing for jvm languages (nlp4j) project foot_2 developed by emorynlp. while this tokenizer is not designed for noisy usergenerated textual data such as sms, several key features make it useful for this setting, such as emoji recognition. lexical data similarly, this can be done across people, conversations, or applications (via keystroke data). post-processing given the sensitive nature of the raw keystroke features, we include a postprocessing script that can be automatically run on the remote aware server (figure 1 ) to remove potentially identifying information. this script uses spacy's named entity recognizer and regular expressions foot_3 to remove mentions of names, numbers, places, etc. the explicit mentions are replaced with their respective category names (e.g., "taylor swift" is replaced with {name}), and the category names are backpropagated through the data to the first keypress in the explicit mention. ## methodological considerations ## case study to date, one study has used the aware-text app with a sample of 120 participants who installed aware-text, answered a series of self-reports, and shared facebook language data. this study compared preexisting social media-based lexical models in their ability to predict self-reports when applied to out-of-domain textual data @xcite . it applied five models trained from facebook language to predict self-reported age, gender, depression, life satisfaction, and stress. each model was separately applied to sms and facebook posts, and the resulting model predictions were compared to self-reports. we report their findings in the results from this study show that for three out of five models, the sms-based predictions did not statistically differ from the facebook-based predictions, indicating that sms is a potential data source for investigating social-psychological traits of people. this paper represents a preliminary analysis, and further investigation is needed into the strengths and weaknesses of sms data. ## comparison of mobile text apps there are several apps which allow for either sms retrieval or keystroke logging. sms retrieval apps are typically designed for personal use, such as data backups and phone transfers, such as sms backup & restore foot_4 , or legal discovery, such as logikcull foot_5 . finally, there are also apps used for survey collection via sms, such as odk foot_6 . these apps send questions and receive answers via sms, and data collection is typically limited to retrieving the survey question responses. keylogging apps are typically designed for the purpose of monitoring, such as kidlogger foot_7 which allows parents to monitor their children's phone activity. thus, most apps which collect this data are not used for general data collection and research purposes. apps which collect sms or keystroke data that are designed for research purposes are typically in the domain of mobile phone-based sensing software. there are several popular apps in this domain which have been used for social scientific research. in ## conclusions aware-text is a novel data collection application for android mobile phones designed to capture textual data through sms and keystrokes. this application allows researchers to collect data from consenting participants at multiple levels of granularity (person, conversation, message, and character) with the additional ability to collect both raw and aggregate lexical measures which preserve privacy. over 85% of americans own smartphones, and a growing number identify as smartphone dependent (i.e., smartphone serves as their primary means of online access). thus, mobile phones offer a rich data source, which can be used as a lens into the daily lives of large sections of the population.