T ranslate L ocally: Blazing-fast translation running on the local CPU (2021)

Abstract: Every day, millions of people sacrifice their privacy and browsing habits in exchange for online machine translation. Companies and governments with confidentiality requirements often ban online translation or pay a premium to disable logging. To bring control back to the end user and demonstrate speed, we developed translateLocally. Running locally on a desktop or laptop CPU, translateLocally delivers cloud-like translation speed and quality even on 10 year old hardware. The open-source software is based on Marian and runs on Linux, Windows, and macOS.

## introduction neural machine translation @xcite is pervasive but has a reputation for high computational cost. the combination of the typically high computational cost, however, has pushed its delivery to the cloud, with a number of cloud providers available @xcite . using a cloud based translation provider carries an inherent privacy risk, as users lose control of their data once it enters the web. potential issues include public disclosure due to not understanding terms of service @xcite , contractors reading user data @xcite , use of user data for advertising, and data breaches. to preserve privacy, we made a translation system that runs locally: translatelocally. once a translation model is downloaded, it does not use an internet connection. running locally is challenging due to a number of factors: the model needs to be small enough to download on a user hardware; translation latency can't be hidden by splitting and parallelising the translation of a large documents across multiple machines; consumer hardware has highly variable computing power; availability of gpu computational resources can't be assumed. we therefore focused on trimming model size and optimising speed for cpus while aiming to preserve translation quality. the result is fast enough that users see translations update as they type with latency comparable to ping times to the cloud. targeting non-expert users, the open-source (primarily mit) software foot_0 is also available as compiled binaries for linux, windows and mac from the official webpage: https:// translatelocally.com . translation models for several language pairs are provided, while advanced users can add their own models. ## design our product is based on the marian machine translation toolkit @xcite , heavily optimised for speed with a qt based gui. ## comparison against existing solutions we compare against two existing desktop machine translation solutions: argos translate foot_5 and opus-cat mt engine @xcite . they both have slightly different use-cases and support different translation languages. we compare bleu scores @xcite on a wmt19 test set @xcite for the english-german language pair, as well as wall-clock and cpu time. we measure only the time necessary for the actual translation. we ignore startup time and issue a translation of an unrelated text before running our test in order to discard any lazy initialisation time. as only translatelocally supports all three platforms, we do pairwise comparison, once on windows for opus-cat vs translatelocally, and once on macos for argos translate vs translatelocally. ## conclusion we presented translatelocally, a desktop translation application, capable of high speed translations on a variety of hardware. our software provides a viable alternative to cloud translation for users who are conscious of their privacy. our product is 100 times faster than competing software and has none of the rate limitations of freemium cloud providers. we start with 10 high quality, optimised models and we aim to continuously add additional language pairs. as our product is open-source and cross-platform, it can be adopted by a wide range of users. the use of marian as a translation engine allows for users to easily train their own models, potentially facilitating internal use for large organizations.