# JoyFlick: Japanese Text Entry Using Dual Joysticks for Flick Input Users

Kaisei Yokoyama, Rei Takakura, Buntarou Shizuki

► To cite this version:

Kaisei Yokoyama, Rei Takakura, Buntarou Shizuki. JoyFlick: Japanese Text Entry Using Dual Joysticks for Flick Input Users. 18th IFIP Conference on Human-Computer Interaction (INTERACT), Aug 2021, Bari, Italy. pp.107-125, 10.1007/978-3-030-85613-7_8. hal-04292388

HAL Id: hal-04292388

https://inria.hal.science/hal-04292388v1

Submitted on 17 Nov 2023

**HAL** is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
<figure><img src="image_2.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
# JoyFlick: Japanese Text Entry Using Dual Joysticks for Flick Input Users

Kaisei Yokoyama¹, Rei Takakura¹, and Buntarou Shizuki¹

University of Tsukuba, Tsukuba, Ibaraki, Japan
{kyokoyama,takakura,shizuki}@iplab.cs.tsukuba.ac.jp

**Abstract.** We show JoyFlick: A Japanese text entry method on game consoles with dual joysticks for flick-input users. To reduce the learning cost, the procedure of JoyFlick is designed based on a flick-input, which is a Japanese text entry method that approximately 80% of young Japanese people use daily on smartphones. The user using JoyFlick is able to enter a Japanese basic phonetic character with two joystick operations. In addition, JoyFlick's widgets spend a little amount of real estate on a screen, making it easy for developers to allocate space for other content. We conducted user studies that compared the performance of JoyFlick with a *Kana* syllabary keyboard, the de-facto standard for the Japanese text entry method on game consoles. The results showed the followings: (1) after training employing 28 phrases (289 characters, mean entry time = 373 seconds), the flick-input users using JoyFlick can enter texts faster than using the *Kana* syllabary keyboard; (2) the flick-input users can enter text using JoyFlick 1.53-fold faster (60.1 CPM) than the *Kana* syllabary keyboard (39.2 CPM) after one-week training in which the users enter texts less than 140 (M = 135) characters per day. These results show that JoyFlick provides a fast and easy Japanese text entry method to the flick-input users using the game controllers and yet spends a little amount of real estate on a screen.

**Keywords:** game controller · dual-joystick input · circular keyboard layout.

## 1 Introduction

Many people have game consoles. The game console users often enjoy text chat on games and share game clips on SNS. In addition, games involving several people are popular [8,9]; participants communicate via text or voice. In such games, users use voice chat or text chat. The use of voice chat is noisy for people around the user limiting the situations in which it can be used. Text chat is quieter than voice chat, although the text entry speed is slower. For these reasons, rapid text entry methods are required on game consoles.

Using a physical QWERTY keyboard is not always a good way for all game console users. To use a physical QWERTY keyboard, the users need to connect hardware keyboards to the game console. In addition, a physical keyboard is
<figure><img src="image_3.png" /><figcaption>**Fig. 1.** Views of game controller (Nintendo Switch Pro Controller [16]). (a) Front view. (b) Top view.</figcaption></figure>

an extra device for game consoles. Thus, the user needs to pay extra costs for installation. On the other hand, the user can save these costs by using a soft keyboard: the user can enter texts using only the game controller (e.g., Fig. 1).

A *Kana* syllabary keyboard is the most popular soft keyboard for Japanese text entry on game consoles. The *Kana* syllabary keyboard has more than 50 keys and 1 cursor. After the user moves the cursor over keys to the desired key, the user pushes the button on the game controller to enter the character. The *Kana* syllabary keyboard has the advantage that any Japanese can understand how to use it at a glance since the *Kana* syllabary keyboard is designed based on a Japanese syllabary chart, which any Japanese learned in elementary school. However, it is difficult to manage many keys with a single cursor, as if a user uses a QWERTY keyboard with only one finger. In addition, since the *Kana* syllabary keyboard has many keys, it occupies almost half of the small game console screen. In such a case, we consider that the keyboard could hinder the user experience.

We built a new Japanese soft keyboard called JoyFlick. In order to become a new option for Japanese text entry on game consoles, JoyFlick was designed to meet the following requirements.

**Simple entry:** With a *Kana* syllabary keyboard, the user moves the cursor to the desired key to enter a character. The relative position of the cursor and the desired key determine the required operations. In most cases, entering a character contains "button-bashing" or long press and then opportune release. If the user can enter texts without these slow operations, the user can enter texts faster.

**Low learning cost:** Many users have been used to enter texts using a *Kana* syllabary keyboard. If the learning cost is lower, more users will use not *Kana* syllabary keyboard but JoyFlick.

**Small widget:** The size of the keyboard should be reduced to that of a *Kana* syllabary keyboard so that the screen is not covered as much as possible.

JoyFlick adopts dual joysticks for text entry. Most modern game controllers have two joysticks (like Fig. 1). In addition, most of the Japanese alphabet (*kana*) consists of 2 elements (consonant and vowel). Therefore, we mapped one joystick
to consonant and the other joystick to vowel. The combination of two selected keys determines the character. JoyFlick adopts the key layout based on flick-input, which approximately 80% of young Japanese people use daily on the smartphone [12]. In addition, JoyFlick has only 15 keys; thus JoyFlick's widget can be smaller than the *Kana* syllabary keyboard.

The four main contributions of this research are:

- We designed a novel Japanese text entry method called JoyFlick using dual joysticks on the game controller.

- We showed that the text entry speed of the flick-input users using JoyFlick exceeds that of a *Kana* syllabary keyboard after 28 phrases of entry by user studies.

- We showed that the flick-input users can enter text using JoyFlick 1.53-fold faster (60.1 CPM) than *Kana* syllabary keyboard (39.2 CPM) after one-week training in which the users entered texts less than 140 (M = 135) characters per day by user studies.

- We showed that the learning cost of JoyFlick for flick-input users is less than the *Kana* syllabary keyboard by user studies.

## 2 Japanese Writing System

<figure><img src="image_4.png" /></figure>

<figure><img src="image_5.png" /><figcaption>**Fig. 2.** Japanese syllabary charts. (Left) Basic letters only. (Right) Voiced letters, the P-sound, and small letters.</figcaption></figure>

This section introduces the basis of Japanese text entry and the Japanese writing system [5]. Japanese text can be transcribed with *kana* and *kanji*. *Kana* are phonograms and *kanji* are ideograms derived from Chinese letters. Each *kanji* letter has (a) phonetic value(s) written using *kana* character(s). In most Japanese text entry systems, the user enters *kana* characters first. Then, the system displays candidates consisted of *kanji* character(s) and *kana* character(s) on the basis of the entered *kana* character(s), since there are multiple candidates because the relationship between *kana* (s) and *kanji* is many-to-many. The user selects the desired candidate to enter the required words that consist of *kana* characters and *kanji* letters (*kana-kanji* conversion).
Kana letters can be transcribed into one to three (M = 1.8) alphabetical letters following the Japanese syllabary chart (Fig. 2). Most basic *kana* letters (“Basic letter” in Fig. 2 left) consist of consonants and vowels (e.g., ‘と’ consists of ‘t’ and ‘o’: ‘t’ as the consonant and ‘o’ as the vowel). Some basic *kana* letters can be converted into the special *kana* letters by adding a symbol (e.g., ‘と’ can be converted into ‘ど’ by adding ‘*’ as a symbol). In other words, the special *kana* letters consist of consonants, vowels, and symbols (e.g., ‘ど’ consists of ‘t’, ‘o’, and ‘*’, Fig. 2 right).

# 3 Related Work

Our work relates to the work on phonetics-based *kana* input (particularly, flick-input), text entry with a game controller, and the hybrid of both.

## 3.1 Phonetics-based Kana Input

Since JoyFlick is a phonetic based *kana* input method, we describe popular methods in the same category used on personal computers, smartphones, and game consoles.

**QWERTY** Japanese speakers usually use a QWERTY keyboard for *kana* input on personal computers. Some of them use a QWERTY keyboard on mobile devices.

A hardware QWERTY keyboard can be connected to game consoles such as Nintendo Switch [15] and PlayStation 4 [20] for *kana* input. However, it is not always a good way for all users due to the following reasons. First, these consoles are not sold with a keyboard; thus, the user needs to purchase a keyboard to use it on a game console. Second, the user must switch from the controller to the keyboard to enter text. Moreover, the user’s posture and position are restricted to where the keyboard is placed. Third, young Japanese people are not very good at typing on a QWERTY keyboard [11]. For these reasons, Japanese people commonly use only the game controller to enter text with a soft keyboard.

<figure><img src="image_6.png" /><figcaption>**Fig. 3.** View of the *Kana* syllabary keyboard.</figcaption></figure>
**Syllabary** Nowadays, the typical text entry method for game controllers is an onscreen selection keyboard [21,22]. The user moves a cursor to the desired key using a joystick or a directional pad (D-pad) and then presses the key using a button. For English text entry, a QWERTY or alphabetical key layout is commonly used [10].

Similarly, an onscreen selection keyboard for Japanese text entry, whose the key layout looks similar to the syllabary chart (Fig. 2), is used. In this paper, we term this “Kana syllabary keyboard” (e.g., Fig. 3). Most Japanese can understand how to use it at the glance because an onscreen selection keyboard displays all characters users can input and because a Kana syllabary keyboard looks similar to the syllabary chart that anyone who has learned the Japanese language knows, which allows Japanese users to quickly recognize where the desired key is.

Although the above design of a Kana syllabary keyboard achieves low learning cost, the design can hinder the user experience. Since a Kana syllabary keyboard has more than 50 keys and the user manages any keys with a single cursor, text entry with the keyboard can be cumbersome because large distances have to be traversed between some keys. Moreover, the keyboard occupies almost half of the screen in some cases since it needs to display many keys.

<figure><img src="image_7.png" /><figcaption>**Fig. 4.** Process of flick-input. (a) Pressing the ‘た’ key will provide feedback on characters that can be input by tapping or flicking the ‘た’ key. (b) Flicking the ‘た’ key downwards will input (c) “と”.</figcaption></figure>

**Flick Input** Flick-input is one of the input methods widely used in touchscreen-equipped mobile devices [5]. Approximately 80% of young Japanese people use flick-input [12]. Flick-input has a 3×4 key layout, similar to the 3×4 key layout of old mobile phones. It adopts four-directional flick operations, which takes advantage of the characteristics of touchscreens and allows for gesture input. The
user can perform five kinds of inputs for each key: tapping the key or flicking in four directions from the center of the key (left, up, right, or down). With this design, the user can easily enter a basic *kana* letter by tapping or flicking a key once. For example, by flicking the ‘た’ key downwards, the user can enter “と” (Fig. 4). Converting a basic *kana* letter to a special *kana* letter requires flicking or tapping the ‘*’ key. For example, flicking the ‘た’ key downwards and then tapping the ‘*’ key once enter “ど”.

**Flick-based Methods** Many techniques are aimed at improving flick input.

Blossom [18,17] supports *kana* input using flick operation on the soft QWERTY keyboard. After pressing the desired key to select the consonant, the user flicks the key to select the vowel. By contrast, JoyFlick's key layout is based on flick-input, not QWERTY.

A method proposed by Ikawa and Miyashita [7] and No-Look Flick [5] adopt flick operations for selecting both consonant and vowel on devices with touchscreens. We extend this approach to dual joysticks.

## 3.2 Text Entry with a Game Controller

Some text entry techniques for game controllers has been developed. PizzaText [24] is an English text entry using dual joysticks. TwoStick [10] is also a text entry method for game controllers with dual joysticks, which is designed to be used for English and many European languages. After selecting a group of characters using a joystick, the user selects the desired character from the group using the other joystick with these methods. We extend this procedure to *kana* input. Moreover, these methods use unique key layouts. We considered that an unfamiliar key layout could increase learning cost.

The text entry technique for game controllers showed by Sandes and Aubert et al. [19] relies on the user's familiarity with QWERTY. Similarly, our keyboard uses the familiarity of young people with flick-input [12] to reduce learning costs.

## 3.3 Phonetics-based Kana Input using a Game Controller

Nakamura et al. [14] used two mice for *kana* input. The user uses each trackball of mice in both hands to select consonants and vowels on a circular menu. The key layout is unique. By contrast, JoyFlick uses joysticks to select slices instead of trackballs.

EGConvert [13] is a *kana* input method using dual joysticks. Two rings that surround a circle share a central point (Fig. 5). Both rings have eight slices. The outer ring, accessed using the left joystick, contains the consonant menu. The inner ring, accessed using the right joystick, contains the vowel menu. The ring arrangement reduces the size of widget, although it may increase the cognitive load on the user to map the left and right joysticks to the outer and inner rings [6]. In addition, the key layout is unique.
<figure><img src="image_8.png" /><figcaption>**Fig. 5.** View of the EGConvert when the user selects ‘と’ (T-o).</figcaption></figure>

These methods use their original key layouts that should be unfamiliar to most users. By contrast, JoyFlick adopts the key layout of flick-input used by approximately 80% of young Japanese to reduce the user’s learning cost.

<figure><img src="image_9.png" /><figcaption>**Fig. 6.** Views of IToNe (a) when both sticks are in the neutral position and (b) when the user selects the ‘と’ (T-o) key.</figcaption></figure>

IToNe [6] is also a *kana* input method using dual joysticks. The left and right half-rings are mapped to the left and right joysticks (Fig. 6). The user tilts one joystick upward to select a consonant and the other joystick downward to select a vowel. The inputs may be in either order. IToNe’s key layout is based on the Japanese syllabary chart while JoyFlick’s one is based on flick-input. In addition, IToNe’s key layout has 5 duplications; that is, each of the 5 vowels has a corresponding key in both left and right bottom half rings. Since reducing these duplications make the slices larger, JoyFlick’s key layout has no duplications.

# 4 JoyFlick

JoyFlick is a novel Japanese text entry method for game controllers. Approximately 80% of young Japanese uses flick-input; thus, JoyFlick adopts the flick-input key layout to save the user’s learning costs. All keys can be selected in one operation. Specifically, to enter a basic *kana* character, the user selects two keys.

## 4.1 Key Layout

JoyFlick adopts a flick-input key layout modified for dual joysticks (Fig. 7). We consider that our key layout enables the flick-input user to save the learning costs.
<figure><img src="image_10.png" /><figcaption>**Fig. 7.** JoyFlick's widget in the neutral position (both sticks) and descriptions.</figcaption></figure>

As described in Section 2, a combination of consonant and vowel determines a basic *kana* letter. Therefore, the consonant selected by one operation and the vowel selected by one operation derives the character. There are 10 consonants and 5 vowels in Japanese. We use joysticks to select the key because a joystick has a rich input vocabulary.

To treat the keys with dual joysticks, we modified flick-input's key layout. Fig. 7 shows the key layout for JoyFlick. The right component and the left component are mapped to the right joystick and the left joystick. The right component corresponds to the consonant key layout of flick-input (Fig. 4c). However, the ‘わ’ key’s position is different from the original: the ‘わ’ key is subscripted at the right of the ‘な’ key. This design gives the user the affordance, telling that the key can be selected with push-in. The left component corresponds to the vowel key layout of flick-input (Fig. 4a). In addition, no elevation angle of the joystick is used for key selection to reduce the required accuracy of operation for the user.

The consonant key layout has more keys than the vowel key layout; thus, the user needs to operate more precisely the former than the latter. We thus mapped the consonant key layout to the right joystick because most people are right-handed.

Special letters are entered by converting basic *kana* letters. JoyFlick adopts the ZR or ZL buttons for the conversion. This design allows the user to quickly convert a character to its special character by keeping their thumbs touching each joystick.

The user pushes the B button to backspace. It is the same button layout as the *Kana* syllabary keyboard to save the user's learning cost.

## 4.2 Entry Method

A *kana* character is entered in two operations. First, the user tilts or presses the right joystick to select the desired consonant. Second, the user tilts or pushes the left joystick to select the desired vowel. When the left joystick returns to neutral, the character is entered. The neutral position of the left joystick corresponds to “no vowel”; the neutral position of the right joystick to the consonant ‘な’ (N). The push-in of the right joystick corresponds to the consonant ‘わ’ (W) and
<figure><img src="image_11.png" /><figcaption>**Fig. 8.** JoyFlick's widget (a) when the user selects the ‘た’ key and (b) when the user selects the ‘た’ and ‘お’ keys to enter “と” (T-o).</figcaption></figure>

the push-in of the left joystick corresponds to the vowel ‘あ’ (a). When a key is selected, the game controller briefly vibrates to give the user tactile feedback to notify the user that a key is selected. The user can enter a special *kana* character by pushing the ZR or ZL button after entering a basic character. For example, pushing the ZR or ZL button one or more times will convert “つ” (T-u) to “づ” (T*-u), “っ” (T~-u), and “つ” (T-u). The user backspaces by pushing the B button.

For example, to enter the character “と” (T-o), the user first selects the consonant ‘た’ (T) by tilting the right joystick to the left (Fig. 8a). After that, the user selects the vowel ‘お’ (o) (in other words, now the user selects the character “と” (T-o)) by tilting the left joystick down (Fig. 8b). In the end, the user enters “と” (T-o) by releasing the left joystick.

## 4.3 Design Features

As a new option of Japanese text entry on game consoles, JoyFlick has the following features:

**Simple entry:** A basic *kana* character is entered in two operations.

**Low learning cost:** JoyFlick's key layout is based on the key layout of flick-input used by approximately 80% of young Japanese people.

**Small widget:** Although a *Kana* syllabary keyboard has more than 50 keys, JoyFlick has only 15 keys except for modifier keys. Thus, the size of JoyFlick's widget is less than half of a *Kana* syllabary keyboard.

# 5 User Study 1

We initially compared the performance of JoyFlick beginners to that of a *Kana* syllabary keyboard (the de-facto standard for Japanese text entry on game consoles).

## 5.1 Participants

We enrolled 24 native Japanese speakers (19–25 years old, M = 21.8, SD = 1.19). 2 participants were female. 1 participant was left-handed, 23 were right-handed,
and none have ambidexterity. 23 participants have a Nintendo Switch [15]. The average total play time of Nintendo Switch is 1.23 × 10³ hours (SD = 1.19 × 10³ hours). No participant had any experience with JoyFlick; all had used the *Kana* syllabary keyboard of Nintendo Switch. On the smartphone, 18 participants use flick-input and the others use QWERTY daily. In this section, we call the former TARGET-USERS and the latter OTHERS.

12 participants joined the study online using Discord [2] as a COVID-19 precaution and the others came to our laboratory (and were thus offline).

## 5.2 Apparatus

<figure><img src="image_12.png" /><figcaption>**Fig. 9.** Display during the tests. In this example, the participant was instructed to enter “いんたらくと” (“INTERACT”) (a) and “いんたらく” (“INTERAC”) has been entered (b). During the trainings, the text explaining how to complete the entry was additionally showed.</figcaption></figure>

We developed a prototype system (Fig. 9), which runs on macOS and Windows 10, in Rust using joycon-rs [23]. We implemented JoyFlick and a *Kana* syllabary keyboard in the system. A Nintendo Switch Pro Controller [16] (Fig. 1) served as the input device, which was connected to the participants’ computer via Bluetooth. We distributed the prototype system to the online participants.

## 5.3 Task

Displayed phrases were transcribed. Entry was completed by pushing the Plus button (Fig. 1). All participants were instructed to enter the text as quickly and accurately as possible. The participants could use the B button to correct errors, although this was not compulsory.
## 5.4 Procedure

Each participant completed two sessions of text transcription. One session was for JoyFlick and the other was for the *Kana* syllabary keyboard. Each session consisted of training and test. We counterbalanced the session and phrase orders to neutralize any effects of learning and fatigue.

We first explained the purpose of the study. We then used a questionnaire to gather participant demographics. Before each session, we explained how to use the relevant keyboard. All participants then transcribed all phrases, which consisted of basic and special *kana* characters and the “prolonged sound” character marks (“—”). We prepared 28 phrases with 289 characters for training and 24 phrases with 167 characters for test. In total, each participant was required to enter the following amount of characters:

$$2 \text{ sessions} \times (289 \text{ characters} + 167 \text{ characters}) = 912 \text{ characters}.$$

All participants completed two sessions within 1 hour.

## 5.5 Results and Analysis

We measured the text entry speed and accuracy. Characters per Minute (CPM) was used as the metric of text entry speed because Japanese text is not written with a space between words. Total Error Rate (Total ER) and Minimum String Distance Error Rate (MSD ER) [1] were used as the metric of text entry accuracy. The methods tested (2 levels) and daily *kana* entry methods into smartphones (2 levels: TARGET-USERS and OTHERS) were the independent variables. The dependent variables were CPM, Total ER, and MSD ER. We used a two-way ANOVA test followed by the post-hoc analysis using Tukey’s HSD test for CPM. Since Shapiro-Wilk tests showed that Total ER and MSD ER had no normality, we used Steel-Dwass tests for Total ER and MSD ER. Only the test data were analyzed.

**Online vs. offline** No statistically significant differences were shown in CPM (Welch’s t-test), Total ER (Wilcoxon-Mann-Whitney test), and MSD ER (Wilcoxon-Mann-Whitney test) between online and offline participants.

**Speed** The text entry speeds are shown in Fig. 10–12. During the tests, the text entry speeds were 42.8 CPM (SD = 8.98 CPM) in JoyFlick and 37.2 CPM (SD = 5.96 CPM) in *Kana* syllabary keyboard; the difference was statistically significant ($p < 0.05, F_{1,44} = 10.4, \eta^2 = 0.166$). Statistically significant differences were shown between TARGET-USERS and OTHERS in JoyFlick ($p < 0.05, d = 1.16$) and JoyFlick and the *Kana* syllabary keyboard in TARGET-USERS ($p < 0.05, d = 1.20$).

JoyFlick was faster than the *Kana* syllabary keyboard for most phrases (Fig. 10). TARGET-USERS’ curve attained 40 CPM in 15 phrases, whereas OTHERS’ curve could not attain 40 CPM in the test (Fig. 11).
<figure><img src="image_13.png" /><figcaption>**Fig. 10.** Average text entry speeds using JoyFlick and *Kana* syllabary keyboard. The gray bands show 95% confidence interval.</figcaption></figure>

<figure><img src="image_14.png" /><figcaption>**Fig. 11.** Average JoyFlick text entry speeds of TARGET-USERS and OTHERS. The gray bands show 95% confidence interval.</figcaption></figure>

<figure><img src="image_15.png" /><figcaption>**Fig. 12.** Average text entry speeds using JoyFlick and *Kana* syllabary keyboard of (left) TARGET-USERS and (right) OTHERS. The gray bands show 95% confidence interval.</figcaption></figure>
**Accuracy** The Total ERs were 8.22% (SD = 3.90%) in JoyFlick and 5.29% (SD = 7.75%) in the *Kana* syllabary keyboard for the tests. Statistically significant difference (*p* < 0.05, *d* = 0.315) was shown between TARGET-USERS using JoyFlick (8.22%, SD = 3.46%) and TARGET-USERS using the *Kana* syllabary keyboard (6.05%, SD = 8.83%). The difference between TARGET-USERS using JoyFlick and OTHERS using the *Kana* syllabary keyboard (3.02%, SD = 1.63%) was also statistically significant (*p* < 0.05, *d* = 1.60). The MSD ERs were 0.648% (SD = 0.965%) in JoyFlick and 0.749% (SD = 0.955%) in the *Kana* syllabary keyboard. The MSD ER differences were not significant. We observed that JoyFlick's Total ER tended to be higher than the *Kana* syllabary keyboard and that JoyFlick's MSD ER tended to be lower than the *Kana* syllabary keyboard.

<table><caption>**Table 1.** The correlation coefficients between total play time; and the CPM, Total ER, and MSD ER.</caption><thead><tr><th></th><th></th><th>CPM</th><th>Total ER</th><th>MSD ER</th></tr></thead><tbody><tr><td rowspan="2">JoyFlick</td><td>TARGET-USERS</td><td>0.321</td><td>-0.415</td><td>-0.041</td></tr><tr><td>OTHERS</td><td>0.480</td><td>0.036</td><td>-0.583</td></tr><tr><td rowspan="2">Kana syllabary keyboard</td><td>TARGET-USERS</td><td>0.155</td><td>0.009</td><td>-0.183</td></tr><tr><td>OTHERS</td><td>-0.256</td><td>0.023</td><td>-0.473</td></tr></tbody></table>

**Total Play Time** The correlation coefficients between the total play time of Nintendo Switch; and the CPM, Total ER, and MSD ER are listed in Tab. 1.

## 5.6 Discussion

**Speed** The results showed that TARGET-USERS, even with little experience, were faster in JoyFlick than the *Kana* syllabary keyboard. In addition, the results showed that the learning cost of JoyFlick for TARGET-USERS is less than for OTHERS: TARGET-USERS learned JoyFlick very quickly.

**Accuracy** The results of MSD ER showed that the text transcripted using JoyFlick was as accurate as using the *Kana* syllabary keyboard. The Total ER and MSD ER suggested that JoyFlick was more error-prone; however, mistakes were easily corrected by backspacing.

**Total Play Time** Overall, the correlations between the total play time and the other metrics were weak, suggesting that the effect of total play time on performance was minimal.

## 6 User Study 2

Participants used JoyFlick and the *kana* syllabary keyboard daily for 7 days.
## 6.1 Participants

We enrolled 15 native Japanese speakers (20-25 years old, M = 21.9, SD = 1.28). All participants were male. 1 participant was left-handed, 14 were right-handed, and none have ambidexterity. All participants owned a Nintendo Switch [15]. The average total play time of Nintendo Switch was 1.52 × 10³ hours (SD = 1.39 × 10³ hours). All participants had been enrolled in the User Study 1. On the smartphone, 11 participants used flick-input and 4 used QWERTY daily. In this section, we call the former TARGET-USERS and the latter OTHERS.

## 6.2 Apparatus, Task, and Procedure

The apparatus, task, and procedure were as described above with the following exceptions:

- All participants completed 1 session daily; both JoyFlick and the Kana syl-labary keyboard were tested; the order of text entry methods was randomly determined.

- We prepared the phrase set that includes 60 phrases with 451 characters for the test.

- All participants entered approximately 18 phrases (i.e., 135 characters) for each method.

- All participants engaged in 1 session once daily for 7 consecutive days.

- After the last session, all participants completed questionnaires about preference.

- In total, each participant was required to enter approximately the following amount of characters:

$$7 \text{ sessions} \times 2 \text{ methods} \times 135 \text{ characters} = 1890 \text{ characters.}$$

All participants completed a session within 15 minutes.

## 6.3 Results and Analysis

The analysis was as described above with the following exceptions:

- The methods tested (2 levels), the daily kana entry methods into smartphones (2 levels: TARGET-USERS and OTHERS), and the number of sessions (7) were the independent variables.

- The main effects were tested via a three-way repeated measures ANOVA for CPM.

- Since Shapiro-Wilk tests showed that Total ER and MSD ER had no normality, we tested Total ER and MSD ER of the last sessions using Steel-Dwass tests.
<figure><img src="image_16.png" /><figcaption>**Fig. 13.** Average text entry speeds for JoyFlick and *Kana* syllabary keyboard. The gray bands show 95% confidence interval.</figcaption></figure>

<figure><img src="image_17.png" /><figcaption>**Fig. 14.** Average JoyFlick text entry speeds of TARGET-USERS and OTHERS. The gray bands show 95% confidence interval.</figcaption></figure>

<figure><img src="image_18.png" /><figcaption>**Fig. 15.** Average text entry speeds for JoyFlick and *Kana* syllabary keyboard of (left) TARGET-USERS and (right) OTHERS. The gray bands show 95% confidence interval.</figcaption></figure>

<figure><img src="image_19.png" /></figure>
**Speed** The text entry speeds are shown in Fig. 13–15. The main effect of sessions was statistically significant ($p < 0.005$, F₁,₁₈₉ = 118, $\eta^2$ = 0.102). A statistically significant difference was evident between JoyFlick and the *Kana* syllabary keyboard ($p < 0.005$, *d* = 7.11). During the last test, the average text entry speeds of TARGET-USERS were 60.1 CPM (SD = 11.8 CPM) in JoyFlick and 39.2 CPM (SD = 7.45 CPM) in the *Kana* syllabary keyboard. Statistically significant differences were evident between TARGET-USERS and OTHERS in JoyFlick ($p < 0.005$, *d* = 0.624) and between JoyFlick and the *Kana* syllabary keyboard of TARGET-USERS ($p < 0.005$, *d* = 2.03).

At the last test, the text entry speed in JoyFlick was 1.45-fold faster than the *Kana* syllabary keyboard (Fig. 13). The text entry speed of OTHERS in JoyFlick overtook that in the *Kana* syllabary keyboard at the first session (Fig. 15 right).

**Accuracy** During the last test, the Total ERs were 11.2% (SD = 6.49%) in JoyFlick and 2.53% (SD = 12.2%) in *Kana* syllabary keyboard. A statistically significant difference was shown between JoyFlick (11.3%, SD = 7.53%) and the *Kana* syllabary keyboard (2.35%, SD = 1.15%) in TARGET-USERS ($p < 0.005$, *d* = 1.60).

For the first test, the MSD ERs were 2.82% (SD = 1.13%) in JoyFlick and 2.11% (SD = 3.30%) in the *Kana* syllabary keyboard. For the last test, the MSD ERs were 1.13% (SD = 8.31 × 10⁻¹%) in JoyFlick and 0.244% (SD = 3.57 × 10⁻¹%) in the *Kana* syllabary keyboard. A statistically significant difference was shown between JoyFlick (1.15%, SD = 0.705%) and the *Kana* syllabary keyboard (0.265%, SD = 0.368%) in TARGET-USERS ($p < 0.05$, *d* = 1.51).

**Questionnaire** The result of the questionnaire showed that all TARGET-USERS and 2 of OTHERS preferred JoyFlick to the *Kana* syllabary keyboard as a Japanese text entry method on game consoles.

## 6.4 Discussion

**Speed** At the end of the study, the text entry speed of TARGET-USERS using JoyFlick was 1.53-fold faster than that of the *Kana* syllabary keyboard (Fig. 15 left). On all tests, the average text entry speed of TARGET-USERS is always approximately 10 CPM faster than OTHERS (Fig. 14). Hence, the learning cost of JoyFlick for TARGET-USERS was still lower than OTHERS since all the participants in User Study 2 participated in User Study 1. In addition, the text entry speed of TARGET-USERS using JoyFlick could have more growth potential than using the *Kana* syllabary keyboard (Fig. 15 left).

**Accuracy** Although the MSD and Total ERs of JoyFlick tended to be higher than those of the *Kana* syllabary keyboard, neither error level would be fatal. During the last test, almost all JoyFlick entry errors were corrected (MSD ER 1.13%). In addition, since the CPM includes the correction time, the results
showed that JoyFlick was faster than the *Kana* syllabary keyboard, even with corrections. We consider that the increased speed (53.3% for TARGET-USERS and 25.0% for OTHERS) outweighs the reduced accuracy (Total ER = 8.64% and MSD ER = 0.886%).

**Questionnaire** TARGET-USERS preferred JoyFlick. OTHERS stated that they would likely to use JoyFlick, which is consistent with the result where the text entry speed of OTHERS in JoyFlick was faster than the *Kana* syllabary keyboard even at the first session.

# 7 Discussion and Future Work

## 7.1 Input Method Editor

While our two user studies showed that JoyFlick has a good text entry performance, incorporating Input Method Editor (IME) into JoyFlick could further improve performance. Japanese text contains several kinds of characters (e.g., *kana*, *kanji*, numbers, symbols, and emoji). The user uses IME, which is a part of an input method, to convert *kana* characters to other kinds of characters. Moreover, a contemporary IME has a feature of autocorrection so that some *kana* entry errors are tolerable. Therefore, IME could compensate for lower accuracy in text entry with JoyFlick. Therefore, evaluating the performance of JoyFlick and the *Kana* syllabary keyboard with IME by user studies is our important future work to examine the feasibility of JoyFlick.

## 7.2 User-defined Key

8 participants of User Study 1 found it difficult to push in the joysticks. 3 participants of User Study 1 suggested that the functions of the left and right joysticks should be swapped. 1 participant of the study suggested user-defined keys would be valuable (this is common in shooting games [3,4]). Therefore, user-defined key mapping might improve user experience.

## 7.3 Other Contexts

JoyFlick was designed for Japanese text entry on game consoles; it could be useful in other contexts. For example, JoyFlick could be used for text entry in virtual reality. While an English text entry method for virtual reality using dual joysticks already exists [24], evaluating the performance using JoyFlick in virtual reality by user studies is left for our future work.
## 7.4 Combination of JoyFlick and Kana Syllabary Keyboard

Since JoyFlick and the *Kana* syllabary keyboard have their strengths, it would be desirable that both are implemented on game consoles. The former would be better when speed is required (e.g., text chat and posting entries of more than two phrases). The latter would be better when accuracy is essential (e.g., SSID and password entry and entering a short phrase such as a name). However, frequent switching between methods increases the cost of text entry. We conjecture that the user chooses a method based on the context of text entry. Presetting the appropriate method for each text entry beforehand could save the cost of switching between different methods.

# 8 Conclusion

We presented a Japanese text entry method on game consoles with dual joysticks for flick-input users, which we call JoyFlick. We designed JoyFlick so that a basic *kana* character is entered in two operations of joysticks. In addition, we adopted the key layout based on that of flick-input used by approximately 80% of young Japanese people to lower the learning cost. Furthermore, JoyFlick has only 15 keys except for modifier keys; thus, JoyFlick occupies the screen space less than half of a *Kana* syllabary keyboard (the de-facto standard for Japanese text entry on game consoles). Our user studies showed that (1) text entry using JoyFlick became faster than the *Kana* syllabary keyboard after training using only 28 phrases; (2) JoyFlick users entered text 1.53-fold faster (60.1 CPM) than the *Kana* syllabary keyboard (39.2 CPM) after one week of training (entry of fewer than 140 characters/day). These results suggest that JoyFlick is a fast, easy, and unobtrusive Japanese text entry method on game consoles.

## References

1. Arif, A.S., Stüerzlinger, W. Analysis of Text Entry Performance Metrics. In: 2009 IEEE Toronto International Conference Science and Technology for Humanity. pp. 100–105. TIC-STH ’09 (2009), https://doi.org/10.1109/TIC-STH.2009.5444533

2. Discord Inc.: Discord — Your Place to Talk and Hang Out. https:// discord.com/, accessed: 2021-01-27

3. Electronic Arts Inc.: Apex Legends - The Next Evolution of Battle Royale - Free on Playstation®, Xbox, and PC. https://www.ea.com/games/apex-legends, accessed: 2021-01-27

4. Fortnite — Free-to-Play Cross-Platform Game - Fortnite: Epic Games, Inc. https://www.epicgames.com/fortnite/, accessed: 2021-01-27

5. Fukatsu, Y., Shizuki, B., Tanaka, J.: No-Look Flick: Single-Handed and Eyes-Free Japanese Text Input System on Touch Screens of Mobile Devices. In: Proceedings of the 15th International Conference on Human-Computer Interaction with Mobile Devices and Services. pp. 161–170. MobileHCI ’13, Association for Computing Machinery, New York, NY, USA (2013), https://doi.org/10.1145/2493190.2493243
6. Go, K., Konishi, H., Matsuura, Y.: IToNe: A Japanese Text Input Method for a Dual Joystick Game Controller. In: Proceedings of CHI '08 Extended Abstracts on Human Factors in Computing Systems. pp. 3141–3146. CHI EA '08 (2008), https://doi.org/10.1145/1358628.1358821

7. Ikawa, Y., Miyashita, H.: "Direction Only" Flick-Input Method for Short-hand with Eyes-Free. In: Proceedings of Interaction 2013. pp. 651–656 (2013), http://www.interaction-ipsj.org/archives/paper2013/data/Interaction2013/interactive/data/pdf/3EXB-22.pdf

8. Innersloth: Among Us for Nintendo Switch - Nintendo Game Details. https://www.nintendo.com/games/detail/among-us-switch/, accessed: 2021-01-27

9. Innersloth: Among Us on Steam. https://store.steampowered.com/app/945360/Among_Us/, accessed: 2021-01-27

10. Költringer, T., Isokoski, P., Grechenig, T.: TwoStick: Writing with a Game Controller. pp. 103–110 (2007), https://doi.org/10.1145/1268517.1268536

11. Kojima, K.: On the Information Literacy of the Net Generation in the BYOD Era. Journal of Nagoya Gakuin University; SOCIAL SCIENCES 52(3), 45–57 (2016), https://doi.org/10.15012/00000619, (in Japanese)

12. Nagasawa, N.: How Japanese University Students Type on Smartphone and PC. Computer & Education pp. 67–72 (2017), https://doi.org/10.14949/konpyutariyoukyouiku.43.67, (in Japanese)

13. Nakamura, K.: Character Input Control Method, Program, Record Media and Character Input Device (2002), https://patents.google.com/patent/JP2002268818A/

14. Nakamura, S., Tsukamoto, M., Shojiro, N.: A Text Input Method Using the DoubleMouse for Wearable Computing Environments. Tech. rep., IPSJ SIG Technical Report (1999), https://ipsj.ixsq.nii.ac.jp/ej/index.php?active_action=repository_view_main_item_detail&page_id=13&block_id=8&item_id=37112&item_no=1

15. Nintendo: Nintendo Switch™ Family – Official Site. https://www.nintendo.com/switch/, accessed: 2021-01-27

16. Nintendo: Pro Controller. https://store.nintendo.com/pro-controller.html, accessed: 2021-01-27

17. Sakurai, Y.: keroxp/Blossom-iOS: SoftwareKeyboard for iOS. https://github.com/keroxp/Blossom-iOS, accessed: 2021-01-27

18. Sakurai, Y., Masui, T.: Blossom - Flick-based Japanese Input System on QWERTY keyboard. Tech. rep., IPSJ (2013), http://id.nii.ac.jp/1001/00094712/

19. Sandnes, F.E., Aubert, A.: Bimanual Text Entry using Game Controllers: Relying on Users' Spatial Familiarity with QWERTY. Interacting with Computers 19(2), 140–150 (2007), https://doi.org/10.1016/j.intcom.2006.08.003, HCI Issues in Computer Games

20. Sony Interactive Entertainment LLC: PS4 — Incredible games, non-stop entertainment — PlayStation. https://www.playstation.com/en-us/ps4/, accessed: 2021-01-27

21. Wilson, A.D., Agrawala, M.: Text Entry Using a Dual Joystick Game Controller. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. pp. 475–478. CHI '06, Association for Computing Machinery, New York, NY, USA (2006), https://doi.org/10.1145/1124772.1124844

22. Wobbrock, J.O., Myers, B.A., Aung, H.H.: Writing with a Joystick: A Comparison of Date Stamp, Selection Keyboard, and EdgeWrite. In: Proceedings of Graphics Interface 2004. p. 1–8. GI ’04, Canadian Human-Computer Communications Society, Waterloo, CAN (2004)
23. Yokoyama, K.: KaiseiYokoyama/joycon-rs: Joy-Con library for Rust. https://github.com/KaiseiYokoyama/joycon-rs, accessed: 2021-01-27

24. Yu, D., Fan, K., Zhang, H., Monteiro, D., Xu, W., Liang, H.: PizzaText: Text Entry for Virtual Reality Systems Using Dual Thumbsticks. IEEE Transactions on Visualization and Computer Graphics 24(11), 2927–2935 (2018), https://doi.org/10.1109/TVCG.2018.2868581