Efficient On-Device Text Simplification for Firefox with Synthetic Data Fine-Tuning (2025)

Abstract: This work presents a system for on-device text simplification that enables users to process sensitive text without relying on cloud-based services. Through the use of quantization techniques and a novel approach to controllable text simplification we reduce model size by up to 75 percent with minimal performance degradation. Our models demonstrate efficient state-of-the-art results using a synthetic dataset of 2909 examples outperforming prior work trained on 300K examples. This efficiency stems from (1) a single control token strategy that precisely targets specific reading levels (2) a contrastive training approach that enriches model understanding through exposure to multiple simplification levels and (3) individual models that dedicate full parameter capacity to specific reading level transformations. Our best models achieve up to 82.18 BLEU at the Advanced level and 46.12 SARI at the Elementary level on standard benchmarks with performance preserved even after aggressive quantization. This work is implemented as a collaboration with the Mozilla AI team to process text entirely locally ensuring sensitive information never leaves the users device. We have a demonstration video https//youtu.be/TzmaxnARMzg and a web demo available at https//pablorom2004.github.io/Simplification-Web-Demo

## introduction text simplification aims to reduce textual complexity while preserving essential meaning, thereby improving accessibility for a broad range of readers @xcite . today, sequenceto-sequence neural models provide state-of-the-art results, but many existing solutions require serverside processing, raising concerns about data privacy and latency when processing sensitive content like medical information or legal documents. second, we create a high-quality synthetic dataset comprising only 2,909 examples generated by carefully prompting large language models. despite its relatively small size, this dataset outperforms the 300k-example wikilarge corpus @xcite on standard benchmarks, highlighting that data quality can far outweigh quantity for efficient model training. we deploy our models locally in the browser via transformers.js foot_0 , ensuring that all data processing happens directly on the user's device. experiments on standard benchmarks show that our models rival or outperform previous approaches, and maintain their quality even when quantized for efficient ondevice operation. ## related work text simplification has progressed from rule-based approaches @xcite @xcite to data-driven methods leveraging parallel corpora @xcite @xcite . for comprehensive overviews of recent developments, see @xcite . pretrained language models like bart @xcite have enabled more fluent and faithful simplifications. control tokens for controllable text simplification were first introduced by @xcite , with @xcite and @xcite further expanding on controllable simplification. @xcite explored different control token configurations and found that tokens representing distinct aspects of complexity (e.g., dependency tree depth, word rank, and length ratio) could effectively guide generation. however, their approach used multiple tokens simultaneously, potentially creating competition for model attention and parameter space. our work builds on this line of research by using a single, level-based control token. synthetic data generation using large language models has emerged as a promising direction for low-resource nlp tasks @xcite . recent work has demonstrated that high-quality synthetic data can match or exceed naturally collected data for various applications @xcite . our work emphasizes quality and diversity over quantity, showing that careful prompt engineering can produce highly effective training examples. on-device nlp has gained traction with growing privacy concerns and the need for offline capability. mozilla's firefox translations project @xcite pioneered browser-based machine translation using onnx format (foundation, 2017) (open neural network exchange, an open standard for machine learning deployment) and webassembly. model compression techniques like distillation and quantization (jain, 2022) have been essential for deploying models in resourceconstrained environments. our work extends these approaches to text simplification, demonstrating successful deployment with minimal performance loss. ## prototype implementation we developed a web-based prototype that demonstrates our text simplification models operating directly in the browser. the implementation uses transformers.js, for all processing locally, ensuring privacy by keeping sensitive text on the user's device. the source code is available at https://github.com/pablorom2004/ simplification-web-demo . ## discussion our experiments reveal several key insights with broader implications. first, our small but highquality synthetic dataset (2,909 examples) outperforms the wikilarge corpus (300k examples), challenging conventional wisdom about data requirements for fine-tuning. this finding suggests that pre-trained models already possess substantial linguistic knowledge and primarily need clear, unambiguous direction rather than extensive examples. our synthetic data succeeds by precisely pointing the model toward the desired task through concise, well-crafted examples that demonstrate the exact transformation patterns required. the clarity and quality of this directional signal prove significantly more valuable than quantity, indicating that carefully engineered llm prompts can create highly effective training data for a wide range of downstream nlp tasks. second, our single control token strategy demonstrates that simplicity can outperform complexity in control mechanisms. by using a single token that directly indicates the target reading level rather than multiple tokens representing different complexity features, we reduce potential parameter competition and make training more efficient for smaller models. third, our finding that individually trained models outperform the control token approach highlights the importance of parameter efficiency in smaller architectures. by dedicating the entire parameter space to learning one specific transformation, these models develop more robust simplification strategies for their target reading levels. finally, the impressive stability of performance across quantization levels (with size reductions up to 75%) indicates that many nlp tasks may not require full floating-point precision. the int8 model's superior performance on readability metrics despite its smaller size suggests that quantization may actually function as beneficial regularization for certain aspects of text simplification. while effective for sentence-level simplification, we found extending to longer contexts or domainspecific text challenging for our bart-base models, suggesting larger architectures may be needed for these scenarios. ## conclusion this work presents an on-device text simplification approach using synthetic data and model quantization that processes text locally in firefox browsers. our contributions include: (1) demonstrating synthetic llm-generated data can outperform much larger human-annotated datasets, ( 2 ) showing specialized models outperform control token approaches for smaller architectures, and