Alignment-Based Decoding Policy for Low-Latency and Anticipation-Free Neural Japanese Input Method Editors (2024)

Abstract: Japanese input method editors (IMEs) are essential tools for inputting Japanese text using a limited set of characters such as the kana syllabary. However, despite their importance, the potential of newer attention-based encoderdecoder neural networks, such as Transformer, has not yet been fully explored for IMEs due to their high computational cost and low-quality intermediate output in simultaneous settings, leading to high latencies. In this work, we propose a simple decoding policy to enable the use of attention-based encoder-decoder networks for simultaneous kana-kanji conversion in the context of Japanese IMEs. We demonstrate that simply decoding by explicitly considering the word boundaries achieves a fairly strong quality-latency trade-off, as it can be seen as equivalent to performing decoding on aligned prefixes and thus achieving an incremental anticipation-free conversion. We further show how such a policy can be applied in practice to achieve high-quality conversions with minimal computational overhead. Our experiments show that our approach can achieve a noticeably better quality-latency trade-off compared to the baselines, while also being a more practical approach due to its ability to directly handle streaming input. Our code is available at https://doi.org/10.5281/ zenodo.11450159.

heading: 1 Introduction
text: Japanese input method editors (IMEs) allow users to input Japanese text, which may include thousands of Chinese characters called kanji, using an unsegmented sequence of a much more limited set of characters such as the kana syllabary or the Roman alphabet (rōmaji) loosely representing its pronunciations, a process commonly referred to as kana-kanji conversion. As kana-kanji conversion is by far the most common approach for inputting Japanese text, Japanese IMEs have become an inte-
gral part of hundreds of millions of people’s daily interactions with mobile devices and computers. Because of their importance, Japanese IMEs have a long history, dating back to 19781, with a lot of attempts at improving them throughout the years. However, despite the progress in traditional techniques, the potential of newer neural network based approaches for IMEs has not yet been fully explored, in spite of them achieving remarkable results in various context-dependent natural language processing tasks by exploiting, e.g., attention-based encoder-decoder networks such as Transformer (Vaswani et al., 2017). This is especially important since the limited phonemic inventory and mora-based phonology of Japanese lead to a very large number of homophones (and consequently homographs when written in kana or rōmaji), making context a vital factor. The above-mentioned gap in the literature can be mainly attributed to two reasons. The first reason, is the noticeably high computational cost of conventional full-sentence encoder-decoder networks when it comes to inference, which makes them unsuitable for providing near real-time output in simultaneous settings. And the second reason, is their potentially low quality intermediate output in simultaneous settings, which can be attributed to train-test mismatch caused by different train and test settings (i.e. offline training vs online testing). In this work, inspired by simultaneous machine translation (SimulMT), we propose a simple decoding policy to enable the use of attention-based encoder-decoder networks for simultaneous kanakanji conversion in the context of Japanese IMEs. We demonstrate that simply decoding by explicitly considering the word boundaries achieves a fairly strong quality-latency trade-off, as it can
1In which Toshiba, JW-10, the first Japanese language word processor, was introduced.
8043
be seen as equivalent to performing decoding on aligned prefixes and thus achieving an incremental anticipation-free conversion by leveraging the monotonic nature of kana-kanji conversion. We further show how such a policy can be applied in practice to achieve high-quality conversions with minimal computational overhead. More specifically, we utilize online word-boundary predictions from an auxiliary linear layer to perform decoding on aligned prefixes, while simultaneously employing an extra auxiliary linear layer incorporated in a wait-k fashion to allow backtracking and corrections in case of mismatches with previous online boundary predictions. Our experiments on conversion quality, in addition to both computational and non-computational (i.e. the lag between source and target) latencies, show that our approach can achieve a noticeably better quality-latency trade-off compared to the baselines, while also being a more practical approach to IMEs due to its ability to directly handle streaming input.

heading: 2 Preliminaries

heading: 2.1 Kana-Kanji Conversion
text: Kana-kanji conversion, the core element of Japanese IMEs, involves converting a hiragana sequence representing the underlying phonetic form, to the corresponding written surface form, consisting of a kana-kanji mixed sequence. The complex and context-dependant relation between kana and kanji, in addition to the lack of delimiters in standard Japanese text and input, makes kanakanji conversion a non-trivial task. For example, the very simple kana sequence “はる” (read as “ha ru”) has more than 4000 potential surface forms based on two different possible segmentations, making educated guesses without considering context fairly challenging. More formally, given an unsegmented sequence of hiragana characters x = (x1, . . . , xn), we want to find the most likely corresponding kana-kanji mixed tokens y = (y1, . . . , ym), m ≤ n, where there is a monotonic many-to-one relation between their alignments. That is, given an alignment function a(s) = t, which maps the hiragana index s to the kanji index twhere ya(s) corresponds to xs, we have:
s1 < s2 =⇒ a(s1) ≤ a(s2) (1)
This monotonic and many-to-one relation is
more clearly illustrated by the non-crossing converging connections in Figure 1, which depicts the alignments between the surface form of an example sentence and its underlying kana sequence. Considering the similarities between the two tasks, we proceed to formulate kana-kanji conversion as a translation from a kana source sequence to the corresponding kana-kanji mixed target sequence, similar to Huang et al. (2018). This interpretation is especially useful as NMT can be seen as the prototypical example of applying encoderdecoder networks to sequence transduction. More specifically, given the simultaneous nature of kana-kanji conversion in IMEs, we will treat it as a case of simultaneous NMT within the prefix-to-prefix framework (Ma et al., 2019).

heading: 2.2 Full-Sentence NMT
text: Conventional full-sentence NMT involves first processing the input sequence x = (x1, . . . , xn) using an encoder before passing it to a decoder, which then greedily predicts the next target token based on both the input sequence and the current partial hypothesis ŷ<t, building the final hypothesis ŷ = (ŷ1, . . . , ŷm) in an autoregressive fashion, where:
ŷ = argmax y P (y | x) (2)
= argmax y
|y|∏
t=1
P (yt | y<t,x) (3)

heading: 2.3 Simultaneous NMT
text: Simultaneous NMT (Satija and Pineau, 2016; Cho and Esipova, 2016; Gu et al., 2017; Ma et al., 2019), in which (partial) translation is generated before reading the entire source sentence, can be seen as a very similar task to (simultaneous) kana-
kanji conversion. Simultaneous MT aims at minimizing latency while achieving the highest quality output possible which are also of importance in kana-kanji conversion. Formally, simultaneous NMT within prefix-toprefix framework (Ma et al., 2019), models the probability P (y | x) as the following:
ŷ = argmax y P (y | x) (4)
= argmax y
|y|∏
t=1
P (yt | y<t,x≤g(t)) (5)
where the monotonic non-decreasing function g(t) denotes a decoding policy, that is, a policy which decides the number of source tokens processed when predicting the target token yt. In other words, at decoding time t, g(t) source tokens must have been READ before conducting a WRITE operation. Generally, the decoding policies are divided into fixed and adaptive policies. Fixed policies, as the name suggests, incorporate a prescribed READ/WRITE decision-making which does not rely on the exact input. One of the simplest and most widely used fixed policies is thewait-k policy (Ma et al., 2019) which initially waits for k input, before starting to generate an output on every new input. Adaptive policies on the other hand, usually rely on complex external agents to decide between READ/WRITE operations based on the input. Adaptive policies constitute most of the early work on simultaneous NMT (Satija and Pineau, 2016; Gu et al., 2017) and they can usually be seen as offering better quality-latency (especially non-computational latency) trade-off at the cost of added complexity. One of the biggest hurdles when it comes to simultaneous MT is anticipation. Anticipation occurs when the model is required to make predictions representing input yet to be seen, forcing it to make predictions based on hallucinated future input, at the cost of translation quality. This is more apparent in Table 1, which illustrates the wait-3 policy failing to keep up with the high fidelity of kanji when applied to kana-kanji conversion, and being forced to anticipate after time step t ≥ 3.

heading: 3 Alignment-Based Decoding Policy

heading: 3.1 Policy Definition
text: As briefly discussed, factors such as high-variance fidelity and different word orders (in case of natu-
ral languages) make avoiding anticipation highly challenging when dealing with different source and target languages. Nevertheless, considering the monotonic and many-to-one nature of alignments between kana and kanji, knowing the word-boundaries is sufficient for aligning both sequences when producing intermediate outputs, and consequently, achieving anticipation-free kanakanji conversion. Figure 1 demonstrates how word-boundaries reveal alignment between those sequences. Leveraging this fact, we propose a wordboundary-based decoding policy for anticipationfree kana-kanji conversion. This policy is depicted in Figure 2, which shows the source-target grid for a sample conversion. As illustrated, in this policy we perform consecutive READ operations until reaching a word boundary, at which we conduct a WRITE operation, eliminating the need for any anticipation.
Following Equation 5, we define our adaptive decoding policy g(t) as:
g(t) = argmin s
( s∑
1
cs = t
) (6)
cs = 1a(s) ̸=a(s+1) (7)
where cs indicates the word boundary status of the corresponding hiragana input xs. Taking the first output y1 in the example illustrated in Figure 2, the corresponding input, which should be up to after the first input including t = 1 boundaries, is x≤2. This is because c1 = 0 and c2 = 1 due to only x2 being a word boundary, which results in g(t) = 2

heading: 3.2 Policy Classifier
text: To obtain word-boundaries ĉ = {ĉ1, . . . , ĉn}, ĉs ∈ {0, 1} in an online fashion at inference time, we define P (c | x) as:
ĉ(0) = argmax c P (c | x) (8)
= argmax c
n∏
s=1
P (cs | x≤s) (9)
However, basing boundary predictions only on past context can make accurate boundary predictions, especially at the beginning of the sentence, challenging. To alleviate that problem by also incorporating future context into our boundary predictions, we can redefineP (c | x) in a wait-k fashion (Ma et al., 2019) as:
ĉ(1) = argmax c P (c | x) (10)
= argmax c
n∏
s=1
P (cs | x≤s+k−1) (11)
Nevertheless, the wait-k boundary predictor cannot be seen as a replacement due to online requirements (i.e. considerable lag between input and output being unacceptable), and thus needs to be incorporated alongside ĉ(0). That is, we train two separate classifiers, ĉ(0), making predictions based on past observations, and ĉ(1), making predictions based on future observations, once observed. In practice, to obtain word-boundaries ĉ(0), we add a linear binary classifier with the weights w(0) ∈ Rdmodel , where dmodel refers to the output dimensions of the encoder, on top of the encoder stack. This layer makes boundary predictions on each input character xs, and if the prediction is
positive, the model will make predictions in accordance with Equation 7 until it reaches an end-ofword token. That is:
ĉ(0)s = 1sigmoid(u(0)(s))≥0.5 (12)
u(0)(s) = w(0) · es + b(0) (13)
where es refers to encoders hidden state output for the input hiragana xs. To calculate ĉ(1), we add another linear layer on top of the encoder, this time with weights w(1) ∈ Rk×dmodel which runs in a wait-k fashion, which can also attend future input tokens, making more accurate predictions. Namely:
ĉ(1)s = 1sigmoid(u(1)(s))≥0.5 (14)
u(1)(s) = w(1) · concat(es:s+k−1) + b(1) (15)
This second classifier is used backtracking and triggering corrections if there are any mismatches between our boundary predictors, in which case the boundary prediction results will be overridden by ĉ(1). Assuming conversion being at the input step xϕ, we have:
ĉs = { ĉ (0) s ϕ− k + 1 < s ≤ ϕ ĉ (1) s 0 ≤ s ≤ ϕ− k + 1
(16)
This correction mechanism is illustrated in Figure 3. In this example, a prediction mismatch is detected for x6 while processing the input x9. This triggers a correction of all the outputs which had attended to x6, shown in crossed gray blocks. As for training, the classifiers are trained in multitask settings, in which the model is trained on the objectives of correct kana-kanji conversions and boundary predictions. That is, we aim at minimizing the following loss function:
L = LNMT + λ · (Lĉ(0) + Lĉ(1)) (17)
whereLNMT,Lĉ(0) andLĉ(1) indicate cross entropy losses for the conversion and boundary predictions respectively, and λ = 20 denotes the weight for the boundary prediction losses.

heading: 4 Experiments and Results
text: In this section, we will review various experiments to compare both the conversion quality and latencies of our model against various baselines such as the previously-mentioned wait-k policy (Ma et al., 2019), which initially waits for k READs before
conducting aWRITE, the re-translation (Arivazhagan et al., 2020) approach, which treats the conversion as a full-sentence offline conversion on each new input, and our modified version of the wait-k policy, which continuously waits for k input before making an output instead of waiting only before the initial output to provide a stronger baseline by better handling the high fidelity of kanji. Unlike offline sentence-level kana-kanji conversion, kana-kanji conversion in the context of IMEs has additional requirements besides the final conversion quality, due to its simultaneous nature. Firstly, non-computational latency, which reflects how closely the output follows the input, is a major factor in evaluation. In addition. the computational cost and the intermediate inference latency caused by it becomes a much bigger factor due to near real-time requirements. Finally, intermediate conversion quality, which can be reflected in non-computational latency metrics, is also of importance for this task.

heading: 4.1 Experiment Settings

heading: 4.1.1 Dataset
text: The Balanced Corpus of Contemporary Written Japanese (BCCWJ) (Maekawa et al., 2014) is chosen as our dataset. BCCWJ can be seen is the standard dataset used for training and evaluating Japanese IMEs, as it provides a rich, balanced selection of various sources, encompassing a wide range of language varieties. Furthermore, BCCWJ provides a relatively large subset of manually reviewed samples which is crucial for an unbiased evaluation, making it one of the very few, if not the only, viable option. As briefly mentioned, the annotations provided
by BCCWJ contain both automatically generated samples alongside samples annotated by humans, which constitutes around 1% of the corpus, labeled as Core. We limit our test data to the Core section of BCCWJ for a more representative evaluation, in addition the size of the test split is reduced to make evaluation computationally feasible. The details about our splits can be found in Table 2. To enable incremental encoding, kana sequences are tokenized as characters, on the other hand for the kana-kanji sequences we apply BPE (Sennrich et al., 2016) on top of BCCWJ’s short unit word (SUW) tokenization to decrease the vocabulary size, which is set to 16000 in our experiments. To be able to continue to use the word-boundary information provided by the corpus, we employ end-of-word suffixes when conducting subword tokenization. These suffixes later help us detect the equivalent multi-stepWRITE operation for each single-step WRITE operation in the case of SUW tokenization.

heading: 4.1.2 Implementation Details
text: Although the word-boundary-based decoding policy can be applied to almost any encoder-decoder network, here we have adopted the Transformer
architecture, as it has been shown to outperform other architectures in various tasks, including NMT. Besides the auxiliary layers, our implementation mostly follows the standard Transformer architecture, with three small differences:
• We use a causal encoder, which allows us to encode the input incrementally without having to recalculate the hidden states of the previous input tokens, and also make training much more efficient (Elbayad et al., 2020).
• We use a cross-attention mask on training so that each target only attends to previous and its own underlying kana sequences to avoid train-test mismatch.
• We adopt a deep encoder and shallow decoder consisting of 10 and 2 layers respectively. This has two benefits: Higher accuracy for our boundary classifiers due to the increased number of available parameters, and less variance in inference time, especially in the case of corrections which can lead to an increased number of decoding steps.

heading: 4.2 Final Conversion Quality
text: For the evaluation of the final conversion quality, we use precision (P ), recall (R), character error rate (CER), and sentence-level accuracy (Accsent) defined as the following, mostly similar to previous work on IMEs (Mori et al., 1998; Tokunaga et al., 2011; Okuno and Mori, 2012):
P = len(lcs(y, ŷ))
len(y) (18)
R = len(lcs(y, ŷ))
len(ŷ) (19)
CER = lev(y, ŷ) len(y)
(20)
Accsent = 1(lev(y,ŷ)=0) (21)
where lcs(·) refers to the longest common subsequence, and lev(·) to the standard Levenshtein distance, both of which are applied on character units, while y and ŷ refer to the reference and predicted kana-kanji mixed sequences respectively. The results for final conversion quality, provided in Table 3, show that the re-translation framework provides the best final conversion quality. This is as expected, considering that the final quality of the re-translation framework is identical to
an offline full-sentence model. On the other hand, the wait-k policy achieves a considerably low conversion quality, something which can be attributed to anticipation caused by the high fidelity of kanakanji mixed tokens. Finally, following the retranslation framework’s results, the modified waitk policy and our approach provide a close and fairly acceptable final conversion qualities.

heading: 4.3 Non-Computational Latency
text: To assess the non-computational latency, which measures how out of sync the input and output sequences are, we use a slightly modified version of revision-aware average lagging (Zheng et al., 2020) as our metric:
RAL(x, ŷ) = 1
τ(|x|)
τ(|x|)∑
t=1
LR(t)− t− 1 r (22)
where τ(|x|) refers to the cut-off step and LR(t) is defined as the source time step s which corresponds to the last revision of tth target token. However different fromZheng et al. (2020), we define r as max(|ŷ|, |y∗|)/|x| instead of |y|/|x|, where y∗ represents the reference prediction. This is to better handle under-/over-generations, which are common in some of our baselines. Referring to the results at Table 3, the wait-k approach, especially at lower k, provides the best latency results, despite a bad conversion quality. This can be attributed to the tendency of wait-k to under-generate, which is caused by too much anticipation due to the high fidelity of the target sequences. Next, we can see the results for our policy followed by the results for the re-translation approach, both of which also offer high-quality conversions. The reason re-translation shows higher latency than our model, is due to its sometimes lower intermediate conversion quality, which can also be reflected in the RAL metric. Finally, we can see that the modified version of wait-k entails quite high latency numbers, as opposed to its good conversion quality.

heading: 4.4 Computational Latency
text: Computational latency is one of themost important factors in IMEs, especially when it comes to ondevice processing. To measure the computational latency, we track the time spent between each READ operation on each test sample, and then calculate the total, maximum, and average time spent between READs based on those. Maximum time
spent is an important factor when it comes to measuring tangible lags by the user, while mean/total times spent provide a basis for evaluating the overall efficacy of the approaches. Table 3 also provides the computational latency of our approach and other baselines. We can see that simultaneous incremental approaches all provide a much more efficient conversion compared to the re-translation approach. While the mean times spent are mostly similar within those approaches, the standard wait-k policy provides the smallest total time spent. With regard to the maximum time spent, our approach demonstrates the best results. Lastly, Figure 4 allows us to better see the latency and quality trade-offs between all the approaches. Here, we can see that both our and the retranslation approaches give very good results for non-computational latency and conversion qualities. However, taking the computational latency into account, our approach provides a better solution with a substantial margin.

heading: 5 Analysis of Boundary Predictors
text: Boundary predictions play a central role in the implementation of our policy, as conversion errors caused by a wrong boundary prediction can propagate to the next tokens and drastically affect the final conversion quality of our model. In this section, we will evaluate the performance of our boundary predictors in isolation, and further analyze its effects on end-to-end results.
Boundary Prediction Quality Table 4 shows the boundary prediction results of our online and wait-k predictors with various k values. We can observe that even for low values of k, the wait-k predictor greatly improves prediction accuracy over the online predictor. Interestingly, the number of average corrections per sentence in the case of using two predictors as discussed previously, stays mostly the same regardless of k, and appears to stabilize for k ≥ 4.
Effects on End-to-End Results While boundary predictor results offer insights, they don’t reveal their impact on overall model performance. We thus proceed to have a look at the end-to-end results of our model concerning our predictors in different settings, and also compare those with the model’s performance on reference boundary data. The results, represented in Table 5, demonstrate that there is a noticeable gap between conversion qualities when using an online predictor versus using the reference boundary data, and that incorporating wait-k predictors alongside the online predictor helps us to greatly close that gap, achieving better conversion quality. As for noncomputational latency, we can observe that incorporating a wait-k predictor for backtracking can actually slightly reduce the latency potentially due to better conversion quality, however, the latency keeps increasing as k increases, overtaking the benefit margin. Finally, as expected, the correction mechanism can increase the computation time, although the effects seem to be quite marginal.
Relative Position of Corrections Figure 5 shows the relative positions of a sentence in which boundary prediction mismatches, and consequently corrections, occur. We can observe that corrections tend to happen mostly uniformly across sentences, with a slightly higher change to happen at the beginning of a sentence, enabling a consistent experience for the end user with regard to possible corrections made.

heading: 6 Previous Work
text: In the conventional approach to Japanese IMEs, kana-kanji conversion is conducted within the noisy channel framework which incorporates a (either statistical or neural) language model, and a translation model, to find the best kana-kanji representation for a kana sequence (Mori et al., 1998; Kudo, 2011; Yao et al., 2018). Besides the conventional framework, other approaches such as using discriminative methods (Tokunaga et al., 2011) and incorporating a joint source channel model (Okuno and Mori, 2012) have been proposed to further improve the results. Although limited to Chinese, more recently there have been some attempts to incorporate modern neural architectures in IMEs. Huang et al. (2018) implements a Chinese IME by formulating pinyin to Chinese conversion as a neural machine translation (NMT) task and uses an attention-based encoder-decoder model. Nonetheless, using a fullsentencemodel and a bi-directional encoder makes it computationally inefficient. Tan et al. (2022) explores using GPT (Radford and Narasimhan, 2018; Radford et al., 2019) for pinyin to Chinese conversion, however, their solutions are also either incompatible with Japanese or computationally inefficient for real-world scenarios.

heading: 7 Conclusion and Future Work
text: In this work we proposed the alignment-based decoding policy for achieving high-quality lowlatency simultaneous kana-kanji conversion using neural encoder-decoder networks. Our results demonstrate that neural encoder-decoder networks can be a viable option for high-quality Japanese IMEs, and we thus encourage more exploration in that area, especially with regards to efficiently adding common IME features such as next-word prediction. In the future, we hope to further address remaining efficacy questions such as memory consumption when it comes to utilizing neural encoderdecoders for IMEs to enablemore viable on-device solutions.

heading: 8 Limitations
text: Although our proposed approach is not limited to Japanese, its advantages might no longer be tangible when applied to other languages due to some language-specific assumptions such as unsegmented input and a many-to-one alignment be-
tween input and output. For example, in the case of some types of simultaneous pinyin to Chinese conversion, we are provided with the segmented input and we can assume a fidelity of one, which could potentially make the conversion possible with a simpler model. Moreover, despite their advantages, neuralnetwork-based approaches (especially approaches based on encoder-decoder architectures) can have some disadvantages compared to conventional methods. For instance, in addition to the higher computational cost, providing customizations such as modifiable vocabulary and online learning are not trivial in deep learning approaches, requiring further investigation regarding their viability.

heading: A User Experience of Japanese IMEs
text: Although most Japanese IMEs provide a similar user experience (UX) for inputting text via kanakanji conversion, the small differences can potentially have great implications on the underlying approach and also the evaluation strategy. Two approaches can generally be found within the design of Japanese IMEs from the view point of UX paradigms, opt-in conversion, in which an IME expects explicit user actions for kana-kanji conversion, and opt-out conversion, in which the aim is to conduct the conversion with as little interruption as possible.
Opt-in Conversion The UX of most common approaches to kana-kanji conversion can be considered to fall into the opt-in conversion paradigm, which heavily relies on explicit user actions.
The opt-in paradigm, does not conduct any implicit conversions on the input, rather it requires
仮名漢字
かな漢字
かなかんじ①
仮名漢字② へんかん
変換
返還
③ 仮名漢字変換
(a) The opt-in approach. Here, in Step 1, the user has to explicitly select a candidate for the input kana to conduct conversion. Step 2 shows the same process repeated for the second chunk to get the final output of Step 3.
① かな漢字変換
かな漢字② 変換
仮名漢字
カナ漢字
③ 仮名漢字変換
(b) The opt-out approach. Here, in Step 1, the conversion is already implicitly applied on top the user input without any explicit actions. Step 2 shows an optional user interaction to fix a hypothetical mistake to get the final output of Step 3.
Figure 6: Kana-kanji conversion within the two different UX paradigms for the user input phrase “かなか んじへんかん” (“ka na ka n ji he n ka n”), meaning “kana-kanji conversion”.
the user to opt in for the desired conversion by explicitly choosing from a candidate list provided separately from the input, as demonstrated in Figure 6a. The opt-in approach encourages the users to conduct conversions in short chunks rather than longer phrases or sentence units, which can result in many manual interactions by the user to conduct the conversion. In addition, conducting conversions in short chunks leads to more limited context for the IME, and subsequently lower quality suggestions.
Opt-out Conversion The alternative paradigm, namely the opt-out conversion, tries to provide a more seamless and interruption-free experience by reducing the required number of user actions. The opt-out conversion paradigm, illustrated in Figure 6b, simultaneously applies the best conver-
sion prediction on top of the input field as the user types without requiring any explicit actions. In this approach the suggestions are usually hidden by default to encourage the user to continue typing uninterrupted while also providing more context to the IME for better conversion, and are only shown by explicit user actions or in situations such as low confidence conversions by the IME (i.e Step 2 in Figure 6b). The opt-out conversion paradigm can lead to a more efficient experience in practice as most conversions will not require explicit corrections, especially that longer context can help the IME provide higher quality conversions. Due to this, we propose our approach with the opt-out paradigm in mind, meaning focusing on sentence-level and top-1 conversion results instead of ranking-based results of shorter chunks.

heading: B Prediction Errors Caused by Our Approach
text: Generally the errors occurring within our approach can be categorized in two main groups, namely:
• Errors caused by wrong boundary predictions
• Errors caused by alternative writing styles
ErrorsCaused byWrongBoundary Predictions This type of error is mostly unique to our approach as it heavily relies on word boundary predictions to conduct conversion. Although subword tokenization appears to allow our approach to recover from some wrong boundary predictions, at points such mistakes lead to actual conversion errors, as shown in the below example:
Source: e え n ん ze ぜ tsu つ wo を chū ちゅう shi し
Predicted Boundaries:
e え n ん ze ぜ tsu つ wo を ch ちゅ u う shi し
Prediction: ... enzetsu 演説 wo を chūshi 中止
(...stop the speech)
Reference: ... enzetsu 演説 wo を chūshi 注視
(...pay close attention to the speech)
in which the model predicts an extra boundary between “ちゅう” (“chū”) and “し” (“shi”), leading to an incorrect conversion for the word “注視”.
Errors Caused by Alternative Writing Styles This type of error is not really unique to our proposed approach but rather originates from the characteristic of the Japanese language, in which one word can have multiple acceptable surface forms (with potentially variable dependency on context). This issue is usually mitigated by presenting the user with a ranked list of conversion candidates instead or alongside the best prediction. Below shows an example of such a conversion error made by our model:
Source: ze ぜ hi ひ ze ぜ hi ひ ha は i い ke け n ん shi し ta た i い
Prediction: zehizehi ぜひぜひ、 haiken shitai 拝見したい...
(...would really like to see...)
Reference: zehizehi 是非是非、 haiken shitai 拝見したい...
...would really like to see...)
here the alternative writing of “是非” has lead to a different prediction compared to the reference. In addition two the twomain groups, another notable error category, although less frequent, consists of words unseen in the training data, particularly proper nouns which can have a much greater degree of irregularities, making it harder for the model to guess the correct surface form based on contextual cues. However, similar to the preceding category, the challenge of out-of-vocabulary (OOV) words is not unique to our approach. This category of errors can potentially be mitigated by standard online learning approaches.

heading: C Training Environment and Hyperparameters
text: In this section we briefly discuss the training environment details and hyperparameters not mentioned in the main body of this paper. For finer details please refer to our code.
Hardware Training was conducted on two 40GB NVIDIA A100 GPUs, totalling around 6.2 GPU hours per model. Although even much shorter training time of around 2.0 GPU hours showed fairly acceptable results in our experiments.
Software Table 6 includes some of the main software and packages used in our work.
Hyperparameters Table 7 represents some of the main details and hyperparameters used for our
training. The choices for hyperparameters are made mostly based on the standard Transformer implementation (Vaswani et al., 2017), task requirements (e.g. deep encoder and shallow decoder for faster inference), andmanual tuning. The values are kept consistent between various models as much as possible to enable direct comparison of evaluation results.

heading: Future Work
text: In the future, we hope to further address remaining efficacy questions such as memory consumption when it comes to utilizing neural encoderdecoders for IMEs to enablemore viable on-device solutions