RE - AIM ing Predictive Text (2021)

Abstract: Our increasing reliance on mobile applications means much of our communication is mediated with the support of predictive text systems. How do these systems impact interpersonal communication and broader society? In what ways are predictive text systems harmful, to whom, and why? In this paper, we focus on predictive text systems on mobile devices and attempt to answer these questions. We introduce the concept of a ‘text entry intervention’ as a way to evaluate predictive text systems through an interventional lens, and consider the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) of predictive text systems. We finish with a discussion of opportunities for NLP.

## introduction predictive text systems were born from an engineering paradigm @xcite , integrated into user interfaces @xcite , and deployed at scale @xcite -with little consideration for their impact on interpersonal communication and broader society. originally designed to increase the speed and ease of communicating with a computer @xcite , predictive text systems have been optimised over time towards the goals of accuracy and efficiency on writing tasks @xcite , and creators of recent email-based predictive text systems highlight how their system saves users over "one billion characters of typing" @xcite . more recently though, @xcite , @xcite , and @xcite respectively highlight concerns for the benefits of predictive text on mobile devices, the effects on the content we write, and the impact of predictive text as it mediates our communications. we aim to begin a discussion on how research priorities and industry practice can change to better our understanding of the benefits and harms of predictive text systems, and to imagine how they might be redesigned. we introduce the concept of a 'text entry intervention' (an intervention delivered as a user enters text) where examples include: an app offering privacy and well-being advice alongside the messages children type @xcite ; a writing assistant intended to influence a writer's grammar, spelling, style and tone @xcite ; or an email assistant intended to reduce repetitive writing and the number of keys we type @xcite . we focus on the common 'text suggestion bar' on mobile devices (figure 1 ) as an instance of a text entry intervention, and describe a path forward for evaluating its impact. note that all of these examples of text entry interventions are real products, being used by real people, with little research into the effects they have on the content produced. once we consider predictive text to be an intervention, we can draw on areas of research focused on the design and evaluation of interventions @xcite @xcite . in doing so, a new set of criteria with which to evaluate predictive text systems becomes available; enabling us to examine aspects of their design and potential impact, and encouraging us to consider the relationships between predictive text systems and their downstream effects (figure 2 ). figure 2 : an example connecting a predictive text system to its impact and harms. the intervention is represented as a point in a design space, which (when combined with other factors) affects: the user's acceptance of the text suggestions and the content they post online, the reader's perception of the content and their behaviour (e.g. like, retweet, or nothing), and (over time) the impact and harm of the intervention. we aim to: (i) begin a discussion about the benefits and harms of predictive text systems, (ii) conceptualise predictive text as an intervention, and (iii) highlight opportunities for nlp research. ## related work predictive text systems have evolved from communication aids designed to benefit novice and reluctant typists @xcite to default features on smartphones @xcite and popular email clients @xcite . while there are some large scale studies of web-based predictive text system for structured transcription tasks @xcite , there is little research into their use in the wild @xcite and evidence that lab-based studies might not reflect real world use @xcite . recent research into the effects of predictive text on the content we write @xcite highlights the need for more studies in-the-wild, but this brings significant challenges @xcite . with regards to evaluating predictive text systems, a key challenge for any researcher-developed intervention is ensuring it changes real world practice. to help intervention developers assess real world impact, the re-aim framework (reach, effectiveness, adoption, implementation, maintenance) was developed @xcite . it asks: does the intervention reach those it should? is it effective in doing 'more good than harm' in real world contexts? @xcite ). is it likely to be adopted and implemented by those required to deliver it? will any changes be maintained over time? the re-aim framework leads to key questions we need to ask to decide how research priorities and industry practice should change @xcite . with regards to improving predictive text systems, we propose re-purposing methods for con-trolling text generation @xcite @xcite @xcite which are being used to control 'bias' in text generation @xcite . with regards to measuring downstream effects, the intersection of causal inference and language provide methods for estimating the causal effects of linguistic properties on downstream outcomes (pryzant et al., 2020); such as the effect of wording, brevity, and tone on message propagation and impact on twitter @xcite and the effects of tone in online debates @xcite . ## evaluating text entry interventions conceptualising predictive text systems as a type of text entry intervention, we draw on the re-aim framework @xcite to consider the reach and effects of predictive text systems and how new interventions might be adopted, implemented, and maintained by the industry actors who control their development and deployment. specifically, application of the re-aim framework (and consideration for how research priorities and industry practice could change) forces us to identify significant gaps in our current knowledge regarding the impact of predictive text systems. ## theory to practice in this section, we consider how we can involve users and affected communities in the design of improved predictive text systems, and how we can engage with industry players. ## opportunities for nlp in this section, we draw on the literature on controllable text generation @xcite @xcite @xcite @xcite and the intersection of causal inference and language @xcite @xcite to discuss the role nlp can play in the development of improved predictive text systems. specifically, we focus on: (1) implementing new systems, and (2) measuring downstream effects. ## conclusion through this work, we hope to begin a discussion about the benefits and harms of predictive text systems, and believe the first step is to conceptualise predictive text systems as interventions. we focused on predictive text systems on mobile devices (the 'text suggestion bar'), but believe research opportunities exists for other nlp-driven text entry interventions. we believe research in this area can benefit from evaluation criteria taken from the field of behaviour change, which focuses on taking interventions from research to practice. we believe there are many opportunities for nlp and hci researchers to improve existing, and design new, predictive text systems. in particular, hci can center the work around the lived experiences of those affected, and nlp can help implement new systems and measure downstream effects. this will be challenging, and involve conducting studies in-thewild and negotiating with the organisations who control the delivery of predictive text systems and the collection of user data.