Proofread: Fixes All Errors with One Tap (2024)

Abstract: The impressive capabilities in Large Language Models (LLMs) provide a powerful approach to reimagine users’ typing experience. This paper demonstrates the Proofread feature in Gboard, a virtual keyboard running on mobile phones. Proofread enables seamless sentence-level and paragraph-level corrections with a single tap. We describe the complete system in this paper, from data generation, metrics design to model tuning and deployment. To obtain models with sufficient quality, we implement a careful data synthetic pipeline tailored to online use cases, design multifaceted metrics, employ a two-stage tuning approach to acquire the dedicated LLM for the feature: the Supervised Fine Tuning (SFT) for foundational quality, followed by the Reinforcement Learning (RL) tuning approach for targeted refinement. Specifically, we find sequential tuning on Rewrite and proofread tasks yields the best quality in SFT stage, and propose global and direct rewards in the RL tuning stage to seek further improvement. Extensive experiments on a human-labeled golden set showed our tuned PaLM2-XS model achieved 85.56% good ratio. We launched the feature to Pixel 8 devices by serving the model on TPU v5 in Google Cloud, with thousands of daily active users. Serving latency was significantly reduced by quantization, bucket inference, text segmentation, and speculative decoding. Our demo could be seen in Youtube.

## introduction gboard is an statistical-decoding-based keyboard on mobile devices developed by google. decoding @xcite is necessary due to the errorprone process of "fat finger" touch input on small screens. according to @xcite , the per-letter error rate is around 8%-9% without decoding. gboard provides various error correction features, some active (automatic) and other passive (require the user's further manual action and selection) to provide a smooth typing experience @xcite . active key correction (kc), and active auto correction (ac), word completions and next-word predictions support the users to type the current word and next word by fixing typos and providing multiple word candidates in the suggestion bar or inline (smart compose). post correction (pc) supports fixing errors in last one or more committed words. furthermore, the more passive spell checker and grammar checker supported by small-sized logistic regression and seq2seq models respectively detect the possible errors in committed sentences and mark them with red underlines, users can fix the errors by clicking the incorrect words and commit the correct words from the displayed candidates. there are two types of user experience limitations with the existing correction approaches. first, users still have to type relatively slowly and accurately to avoid making too many or too severe errors that the small (but instantly fast) on-device correction models such as kc, ac and pc cannot handle due to their limited ability to model longer-span context. second, users need to manually engage in the multi-step passive correction features, such as the grammar checker and the spell checker, to correct the committed words one after another. supervising the committed words while typing and fixing errors sequentially by editing after commit take users' cognitive and visual-motor resources and slow down their typing speed. one desired pattern of fast typing users of gboard is to focus on keyboard only without checking the committed words while typing. to this end, a high quality sentence-or-higher-level correction feature is often called for, in order to help those "fast and sloppy" users who prefer to focus on typing then 1 286 switch to error corrections at a higher level. in this paper, we propose the proofread feature to alleviate the pain points of fast typers by providing the sentence-level and paragraph-level error fixes with only one-tap. proofread falls into the area of grammatical error correction (gec), which has a long history of research from rulebased to statistical approaches to neural network models @xcite . the astonishing capability growth of large language models (llms), offers a new opportunity to unlock the high quality sentence-level grammar fixes. we present the entire system to tune and serve the llm model behind proofread in this paper. the system consist of four parts, data generation, metrics design, model tuning and model serving. firstly, dataset is generated by a carefully designed error synthetic framework which integrates errors frequently made on keyboard to simulate the users' input, several further steps are conducted to ensure the data distribution is close to gboard domain maximally. secondly, several metrics are designed to measure the model from various dimensions. as the answers are always not unique specifically for long examples, the metric combined with grammar error existence check and same meaning check based on llms are considered as the key metrics for comparing the model quality. thirdly, inspired by instructgpt @xcite , supervised fine-tuning followed by the reinforcement learning (rl) tuning is adopted to obtain the llm dedicated for proofread feature. results suggested that our rewrite task tuning and reinforcement learning recipe significantly improves the proofreading performance of the foundation models. to reduce the serving cost, we build our feature on top of the medium sized llm palm2-xs, which could be fit int a single tpu v5 after 8-bit quantization. we further optimize latency with bucket keys, segmentation and speculative decoding @xcite . our model now is launched to benefit thousands of users with pixel 8 devices. figure 1 exhibits our model quality on one extreme corrupted case from andrej karpathy 1 , which indicates our tuned model is strong enough to handle various of heavy typo errors made by users. the contribution of this paper can be summarized as follows: ## 288 dataset the upper half of figure 2 illustrates the pipeline to generate the dataset. we initially sample data from the web crawled dataset, which is then processed by a gec model to fix the grammar errors. each item in the dataset consists of a source sentence with several possible reference sentences. grammar errors are then synthesized into the source sentence to simulate users' inputs, various kinds of errors which frequently happen in gboard real scenarios are involved in this step, including: the last step is to filter the noise data by utilizing llm with careful designed instructions to avoid polluting the model. data is diagnosed by various dimensions, including: an example of the synthetic dataset is showcased below: source: "good moning! hey si, how. a u dou?" reference1: "good morning! hey sir, how are you doing?" reference2: "good morning! hey sister, how are you doing?" moreover, part of the examples labeled by human rater are used as the golden set for evaluation. ## metrics it's of key importance to define the correct metrics which are aligned to user experiences online before the feature goes to public. in this section, several metrics are designed to measure the model quality. given the three elements, input (corrupted text), answer (predicted candidate from the model) and target (ground truth), we present the following metrics. ## model tuning the lower half of figure 2 illustrates the tuning steps of the model for proofread. we start from instruction-tuned models. palm2-xs model from @xcite is the candidate model. ## model serving google's tpuv5e @xcite is utilized to serve the proofread model, which is the latest google tpu chip with 16gb hbm. 8-bit quantization is facilitated to reduce the memory footprint and latency without observing quality degradation. in the context of our research, which predominantly focuses on deployment within chat applications, it has been observed that the average sentence length seldom exceeds 20 words. consequently, we have established a discrete set of bucket keys, specifically @xcite @xcite , to categorize the input data accordingly. furthermore, we have calibrated the temperature parameter to a value of 0.3, aiming to maintain a constrained level of creativity in the proofreading outcomes, thereby ensuring relevance and coherence. to be capable of handling more extensive documents, a systematic approach is employed wherein the document is segmented into individual paragraphs. all paragraphs are then processed in parallel, allowing for a more manageable and efficient analysis. decoding latency (ms) baseline 314.4 + speculative 190.6 (-39.4%) additionally, our methodology incorporates the use of speculative decoding @xcite , complemented by heuristic drafter models that are tailored to align with user history patterns. under our proofreading case, the initial input would naturally fit into the speculative draft so external drafter models are needed. we share an example to illustrate the process in figure 3 . this innovative approach significantly contributes to the reduction of operational costs. through empirical evaluation in ## conclusions this paper presents a novel proofread feature implemented within gboard, powered by a carefully refined llm. our work demonstrates the significant potential of llms to enhance the users' typing experiences by providing high-quality sentenceand paragraph-level corrections. we detailed our comprehensive approach, encompassing synthetic data generation pipeline aligned with real-world use cases, multifaceted metrics design, two-stage model tuning (multiple phased sft followed by rl) and the efficient model deployment. specifically, our findings reveal that rewrite task tuning benefited the sft model by enhancing the meaning alignment ability of the model. additionally, we discovered the value of global and direct rewards during rl tuning, which could further improve the model by reduce grammar errors significantly. rigorous experiments demonstrated that our tuned palm2-xs model achieved an impressive 85.56% good ratio and 14.44% bad ratio. the successful deployment of the model on tpu v5, leveraging optimizations such as quantization, bucket inference and speculative decoding, highlights its real-world viability. this work underscores the transformative power of llms in the realm of user input experiences. future research directions include leveraging real-user data, multilingual adaption, personalized assistance for diverse writing styles and privacy-preserving on-device solutions. this technology has the potential to fundamentally improve how we interact with our devices.