When does text prediction benefit from additional context? An exploration of contextual signals for chat and email messages (2021)

Abstract: Email and chat communication tools are increasingly important for completing daily tasks. Accurate real-time phrase completion can save time and bolster productivity. Modern text prediction algorithms are based on large language models which typically rely on the prior words in a message to predict a completion. We examine how additional contextual signals (from previous messages, time, and subject) affect the performance of a commercial text prediction model. We compare contextual text prediction in chat and email messages from two of the largest commercial platforms Microsoft Teams and Outlook, finding that contextual signals contribute to performance differently between these scenarios. On emails, time context is most beneficial with small relative gains of 2% over baseline. Whereas, in chat scenarios, using a tailored set of previous messages as context yields relative improvements over the baseline between 9.3% and 18.6% across various critical service-oriented text prediction metrics.

## introduction email and chat communication tools are increasingly important for completing daily professional and personal tasks. given the recent pandemic and shift to remote work, this usage has surged. the number of daily active users in microsoft teams, the largest business communication and chat platform, has increased from 20 million @xcite to more than 115 million in october (2020). on the other hand, email continues to be the crucial driver for formal communication showing ever increasing usage. providing real-time suggestions for word or phrase auto-completions is known as text prediction. the efficiency of these communications is enhanced by suggesting highly accurate text predictions with low latency. text prediction services have been deployed across popular communication tools and platforms such as @xcite or gmail smart compose @xcite . modern text prediction algorithms are based on large language models and generally rely on the prefix of a message (characters typed until cursor position) to create predictions. we study to what extent additional contextual signals improve text predictions in chat and email messages in two of the largest commercial communication platforms: microsoft teams and outlook. our contributions are summarized as: the remainder of the paper is organized as follows. we give an overview of state-of-the-art related research in section 2. more details on the signals used for contextualization are provided in section 3. section 4 provides information on the language model, performance metrics, and statistical details about the data. experiment results and comparisons are presented in section 5. we conclude in section 6. ethical considerations on the data and processes are discussed in section 7. ## related work text prediction services have been applied for various applications, including text editor @xcite , query autocompletion on search engine @xcite , mobile virtual keyboard @xcite . recently prediction service is applied on communication tools for composing email and chat messages to improve user writing productivity @xcite @xcite . to predict correct text continuation, such applications leverage efficient lookups with pregenerated candidates, using most popular candidates (mpc) @xcite , or using large-scale language models @xcite . state-of-the-art language models @xcite @xcite rely on the most recent deep learning architectures, including large lstms @xcite or transformers @xcite , while prior approaches involve n-gram modeling @xcite @xcite . in this work, we focus on the application of text prediction on production-level online communication tools, to help users compose emails @xcite , and in addition chat messages. in particular, we focus on examining useful contextual signals to give more accurate predicted text, using time, subject, and prior messages. various contextualization techniques (e.g., hierarchical rnns) have been applied to add useful additional signals such as preceding web interaction, linking pages, similar search queries or visitor interests of a page @xcite ; previous sequence of utterances @xcite @xcite or related text snippets @xcite . ## contextualization concepts we examine several signals accompanying the main message text: message compose time, subject, and previous messages. we combine these signals with the message body into a single "contextualized" string, using special tokens to separate signals, as shown in figure 1a . this approach is inspired by @xcite , as they showed that concatenating contextual signals into a single input string gave a comparable result to more complex methods encoding these signals separately foot_0 . the remainder of this section explains details about each contextual signal we use. time composition time is a contextual signal which can provide added value for text prediction, enabling suggestions with relevant date-time words, like "weekend", "tonight". we encode local date and time, as shown in figure 1a , and @xmath0. subject message subjects often contain the purpose or summarized information of a message. in the email scenario, we use subject as context. in the chat scenario, subject is not available, so we use the chat window name as a proxy for subject (can be auto-generated or manually set by users). in both cases, the subject context is wrapped @xmath1. previous email messages previous messages can provide valuable background information which influences the text of the current message being composed. in the email case, we create pairs of messages and replies. these pairs are concatenated with @xmath2. in cases where the email was the first in a thread, the prior email context is left blank. previous chat messages prior message contextualization for chat scenario is much more complex. chat conversations typically consist of many small messages sent in quick succession. given the email and chat message length statistics in section 4, we expect chat messages to be about 10× smaller than emails. so, we limit chat histories to 20 messages, which is roughly equivalent to an email-reply pair in length. among these prior messages, any number and any order could be from the current sender, or the other participant. we segment chat histories by message blocks and time windows. a series of uninterrupted messages sent by one sender is considered as a single message block. messages sent within the past n minutes are within a time window, which enforces recency as a proxy for relevance. we define three prior message context aggregation modes in the chat scenario (visualized in figure 1b ), mimicking prior email context: for each mode, we consider time windows of @xmath3, 5, 10} minutes. 4 data and language model 4.1 data our model training depends on real messages from two of the largest commercial communication platforms microsoft teams and outlook; this involves a multi-pronged system for ensuring our customers' ## experiments and results we conduct experiments for both email and chat messages with individual contextual signals (time, subject, prior messages) and combinations of those. ## conclusions we study the role of context in text prediction for chat and email platforms. testing with previous messages, subject, time as additional contextual signals, we find that the different characteristics of emails and chat messages influence the selection of contextual signals to use. previous message contextualization leads to significant gains for chat messages from microsoft teams, when using an appropriate message aggregation strategy. by using a 5 minute time window and messages from both senders, we see a 9.4% relative increase in the match rate, and an 18.6% relative gain on estimated characters accepted. chat messages are often short and can lack context about a train of thought; previous messages can bring necessary semantics to the model to provide a correct prediction. benefits are comparatively insignificant for subject and compose time as contextual signals in chat messages. in the email scenario based on microsoft outlook, we find that time as a contextual signal yields the largest boost with a 2.02% relative increase on the match rate, while subject only helps in conjunction with time, and prior messages yields no improvement. more complex models may be needed to reap subject and prior message gains for emails, but the current architecture was chosen for largescale serving latency. future work involves exploring different encodings for contextual signals, such as utilizing hierarchical rnns @xcite to better capture context, or using more advanced architectures such as transformers or gpt-3.