Alignment-Based Decoding Policy for Low-Latency and Anticipation-Free Neural J apanese Input Method Editors (2024)

Abstract: Japanese input method editors (IMEs) are essential tools for inputting Japanese text using a limited set of characters such as the kana syllabary. However, despite their importance, the potential of newer attention-based encoder-decoder neural networks, such as Transformer, has not yet been fully explored for IMEs due to their high computational cost and low-quality intermediate output in simultaneous settings, leading to high latencies. In this work, we propose a simple decoding policy to enable the use of attention-based encoder-decoder networks for simultaneous kana-kanji conversion in the context of Japanese IMEs inspired by simultaneous machine translation (SimulMT). We demonstrate that simply decoding by explicitly considering the word boundaries achieves a fairly strong quality-latency trade-off, as it can be seen as equivalent to performing decoding on aligned prefixes and thus achieving an incremental anticipation-free conversion. We further show how such a policy can be applied in practice to achieve high-quality conversions with minimal computational overhead. Our experiments show that our approach can achieve a noticeably better quality-latency trade-off compared to the baselines, while also being a more practical approach due to its ability to directly handle streaming input. Our code is available at https://anonymous.4open.science/r/transformer_ime-D327.

## introduction japanese input method editors (imes) allow users to input japanese text, which may include thousands of chinese characters called kanji, using an unsegmented sequence of a much more limited set of characters such as the kana syllabary or the roman alphabet (rōmaji) loosely representing its pronunciations, a process commonly referred to as kana-kanji conversion. as kana-kanji conversion is by far the most common approach for inputting japanese text, japanese imes have become an inte-gral part of hundreds of millions of people's daily interactions with mobile devices and computers. because of their importance, japanese imes have a long history, dating back to 1978 foot_0 , with a lot of attempts at improving them throughout the years. however, despite the progress in traditional techniques, the potential of newer neural network based approaches for imes has not yet been fully explored, in spite of them achieving remarkable results in various context-dependent natural language processing tasks by exploiting, e.g., attention-based encoder-decoder networks such as transformer @xcite . this is especially important since the limited phonemic inventory and mora-based phonology of japanese lead to a very large number of homophones (and consequently homographs when written in kana or rōmaji), making context a vital factor. the above-mentioned gap in the literature can be mainly attributed to two reasons. the first reason, is the noticeably high computational cost of conventional full-sentence encoder-decoder networks when it comes to inference, which makes them unsuitable for providing near real-time output in simultaneous settings. and the second reason, is their potentially low quality intermediate output in simultaneous settings, which can be attributed to train-test mismatch caused by different train and test settings (i.e. offline training vs online testing). in this work, inspired by simultaneous machine translation (simulmt), we propose a simple decoding policy to enable the use of attention-based encoder-decoder networks for simultaneous kanakanji conversion in the context of japanese imes. we demonstrate that simply decoding by explicitly considering the word boundaries achieves a fairly strong quality-latency trade-off, as it can be seen as equivalent to performing decoding on aligned prefixes and thus achieving an incremental anticipation-free conversion by leveraging the monotonic nature of kana-kanji conversion. we further show how such a policy can be applied in practice to achieve high-quality conversions with minimal computational overhead. more specifically, we utilize online word-boundary predictions from an auxiliary linear layer to perform decoding on aligned prefixes, while simultaneously employing an extra auxiliary linear layer incorporated in a wait-k fashion to allow backtracking and corrections in case of mismatches with previous online boundary predictions. our experiments on conversion quality, in addition to both computational and non-computational (i.e. the lag between source and target) latencies, show that our approach can achieve a noticeably better quality-latency trade-off compared to the baselines, while also being a more practical approach to imes due to its ability to directly handle streaming input. ## experiments and results in this section, we will review various experiments to compare both the conversion quality and latencies of our model against various baselines such as the previously-mentioned wait-k policy @xcite , which initially waits for k reads before s u conducting a write, the re-translation (arivazhagan et al., 2020) approach, which treats the conversion as a full-sentence offline conversion on each new input, and our modified version of the wait-k policy, which continuously waits for k input before making an output instead of waiting only before the initial output to provide a stronger baseline by better handling the high fidelity of kanji. unlike offline sentence-level kana-kanji conversion, kana-kanji conversion in the context of imes has additional requirements besides the final conversion quality, due to its simultaneous nature. firstly, non-computational latency, which reflects how closely the output follows the input, is a major factor in evaluation. in addition. the computational cost and the intermediate inference latency caused by it becomes a much bigger factor due to near real-time requirements. finally, intermediate conversion quality, which can be reflected in non-computational latency metrics, is also of importance for this task. ## analysis of boundary predictors boundary predictions play a central role in the implementation of our policy, as conversion errors caused by a wrong boundary prediction can propagate to the next tokens and drastically affect the final conversion quality of our model. in this section, we will evaluate the performance of our boundary predictors in isolation, and further analyze its effects on end-to-end results. model p r f 1 acc sent #cor online 93.92 94.02 93.77 20.90 0 @xmath0.63 98.27 98.37 63.20 2.184 @xmath1.48 99.48 99.45 86.75 2.352 @xmath2.58 99.63 99.58 89.90 2.363 ## previous work in the conventional approach to japanese imes, kana-kanji conversion is conducted within the noisy channel framework which incorporates a (either statistical or neural) language model, and a translation model, to find the best kana-kanji representation for a kana sequence @xcite . besides the conventional framework, other approaches such as using discriminative methods @xcite and incorporating a joint source channel model @xcite have been proposed to further improve the results. although limited to chinese, more recently there have been some attempts to incorporate modern neural architectures in imes. @xcite implements a chinese ime by formulating pinyin to chinese conversion as a neural machine translation (nmt) task and uses an attention-based encoder-decoder model. nonetheless, using a fullsentence model and a bi-directional encoder makes it computationally inefficient. @xcite explores using gpt @xcite for pinyin to chinese conversion, however, their solutions are also either incompatible with japanese or computationally inefficient for real-world scenarios. ## conclusion and future work in this work we proposed the alignment-based decoding policy for achieving high-quality lowlatency simultaneous kana-kanji conversion using neural encoder-decoder networks. our results demonstrate that neural encoder-decoder networks can be a viable option for high-quality japanese imes, and we thus encourage more exploration in that area, especially with regards to efficiently adding common ime features such as next-word prediction. in the future, we hope to further address remaining efficacy questions such as memory consumption when it comes to utilizing neural encoderdecoders for imes to enable more viable on-device solutions.