The Internals of the Senn Japanese Input Method
Tags: Architecture, Inputmethod, Japanese

Introduction

Senn is a Japanese input method I am currently developing. I have tried to make the design and the implementation of Senn reflect the domain of Japanese input as my personal experiment of Domain-Driven Design. In this story, I will describe the domain of Japanese input of my understanding and how I implemented Senn through it.

For those who have used a Japanese input method, I will only describe a input method of type Latin-character-input (ローマ字入力).

Why Japanese Input Method?

In this section, I introduce a problem in inputting Japanese characters to a machine and how Japanese input methods has been resolving this problem. I will explain the problem and how the problem has been resolved in the first place because these two things relate to important decisions such as classifying and naming objects, data, and functions, and thus the most important source for the code generated later.

The Japanese language has more than 3,000 kana and kanji characters, but a common keyboard has only around 100 keys. It is impossible or extremely impractical to map each Japanese character directly to a key in a keyboard. We need something other than keyboard that enables us to input Japanese characters to a computer. Japanese input method is one such thing.

Japanese input method accepts an input sequence of keys from a keyboard, transforms the sequence to the corresponding pronunciation of kana characters and convert it to a sequence of kana and kanji characters which has the same pronunciation of the input. The pronunciation the user inputs typically is not for an individual Japanese character, but for a sequence of Japanese words in a sentence that the user has in mind. Thus, using the input method, Japanese characters are inputted to a computer through conversions from pronunciations to words of Japanese sentences.

I will explain how a user inputs a Japanese word to a computer with a Japanese input method. Let’s suppose that the user wants to input the word “東京” (Tokyo) to the computer. To do this with the input method, the user first inputs the key sequence “toukyou.” The input method accepts the sequence and shows the pronunciation “とうきょう” on the screen. Here, the input method accepted the input sequence “toukyou” and transformed it to the corresponding pronunciation “とうきょう” as mentioned above. Transforming from “toukyou” to “とうきょう” is done by a rule that maps a sequence of Latin characters (“to”, for example) to Japanese kana characters (“と”, in this example).

After checking that the pronunciation shown on the screen is right for the word in mind, the user starts conversion by hitting the space key. Hitting the space key is a typical sign for starting conversion. The input method search words (or word sequences generally) that have the pronunciation “とうきょう”, and shows the conversion candidate “東京” (Tokyo), whose pronunciation is of course “とうきょう.” Then the user commits the candidate because it is the word that the user wants to input to the computer.

While a user is inputting keys to prepare a pronunciation, the input method transforms the each sequence of keys to the corresponding pronunciation, and shows it on the screen. In the above example in which the key sequence is “toukyou”, the user first hits “t”, and the input method shows “t” on the screen. Second, the user hits “o”, and the input method shows not “to” but “と” because “to” is mapped to the pronunciation of “と.” The next input is “u”, and “とう” is shown. Next “k”, then “とうk”, and so on. After the user inputs the second “u” of “toukyou”, the input method shows “とうきょう.”

This example is a simple one where the user inputs a single Japanese word. In most cases, however, a user usually wants to input several words since the user have a sentence composed of those words in mind rather than a single word. Japanese input methods can handle this by just accepting a key sequence for the sentence pronunciation, which is just a concatenation of the pronunciation of each word. For example, “東京に行った” (I went to Tokyo) is a sentence composed of multiple words, where the pronunciation is “とうきょうにいった.” When a user wants to input this sentence, first the user inputs the key sequence “toukyouniitta”, which is then transformed to the pronunciation “とうきょうにいった.” After the space key is hit, the input method shows the conversion result “東京|に|行った” on the screen, where the vertical line “|” indicates a word boundary. Since the conversion result is what the user wants to input to the computer, the user commits the result.

Because some Japanese words have the same pronunciation, mapping from a pronunciation to a Japanese word is one-to-many. A user may see more conversion candidates for each pronunciation segment if the first conversion candidates are not the user wants.

The Requirements

As mentioned in the previous section, a Japanese input method is required to play several behaviors.

The input method should accept a key sequence from a user, transform it to the corresponding pronunciation, and convert the pronunciation to a sequence of Japanese words of kana-kanji characters.

In addition to the capability as a standard input method above, I want to add this requirement: improvement of conversion accuracy by those who have a deep knowledge of Japanese language modeling or user input texts.