# Character Input in Augmented Reality: An Evaluation of Keyboard Position and Interaction Visualisation for Head-Mounted Displays

Maite Frutos-Pascual, Clara Gale, Jake M. Harrison, Chris Creed, Ian Williams

► To cite this version:

Maite Frutos-Pascual, Clara Gale, Jake M. Harrison, Chris Creed, Ian Williams. Character Input in Augmented Reality: An Evaluation of Keyboard Position and Interaction Visualisation for Head-Mounted Displays. 18th IFIP Conference on Human-Computer Interaction (INTERACT), Aug 2021, Bari, Italy. pp.480-501, 10.1007/978-3-030-85623-6_29. hal-04331582

HAL Id: hal-04331582

https://inria.hal.science/hal-04331582v1

Submitted on 8 Dec 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
<figure><img src="image_2.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
# Character Input in Augmented Reality: An evaluation of keyboard position and interaction visualisation for Head-Mounted Displays

Maite Frutos-Pascual¹[0000-0001-5861-1259], Clara Gale¹, Jake M. Harrison¹, Chris Creed¹, and Ian Williams¹[0000-0002-0651-0963]

DMT Lab, School of Computing and Digital Technology,
Birmingham City University, United Kingdom
maite.frutos@bcu.ac.uk

**Abstract.** Character input in immersive environments is a non trivial task that has attracted much attention in recent years. This paper presents an evaluation of keyboard position, orientation and interaction together with the influence of visual interaction feedback in a controlled character input task with 27 participants in Augmented Reality (AR). It presents 5 different keyboard locations (3 bounded to the headset and 2 bounded to the non-dominant hand of the user) and 3 visual interaction feedback methods (finger raycast, fingertip glow and both combined). Objective (completion time, accuracy, Key per Minute (KPM)) and subjective (After Scenario Questionnaire (ASQ)) metrics are presented. Results showed that keyboard placement had an effect on accuracy, KPM metrics and subjective preference, with keyboard visualisation parallel and bounded to the headset position and orientation outperforming other keyboard locations.

**Keywords:** Augmented Reality · Text Input · Usability · User Evaluation · Visualisation

## 1 Introduction

Text input is a non-trivial [31] and fundamental task [7] which is an integral component of interaction in immersive environments. Subsequently text input has recently received increased attention, notably in Virtual Reality (VR), with authors focusing on interaction visualisation challenges [13, 14, 21, 34], typing performance [7, 29] and feedback methods [26, 28], among others. Typing in Augmented Reality (AR) has also been explored, with the key focus on the use of mid-air gestures [22], combined input mechanisms [8, 40] and keyboard representations [22].

Multi-modal approaches have been extensively evaluated, with speech recognition being a major input method for Head-Mounted Displays (HMDs). However, this presents environmental limitations such as noise, social acceptance [22] and privacy concerns for sensitive information (i.e. passwords and personal messages). Alternative approaches suggest the use of head-gaze and eye-tracking interaction combined with dwell time and click interaction [33] or touch gestures [1] for typing. While these offer hands free interaction, they have been deemed to be constrained and present challenges associated to performance, user strain and motion sickness [33].
As an alternative, hand-held controllers are presently one of the most common interaction paradigms in VR [40] with existing consumer-grade HMDs often relying on indirect pointing mechanisms via hand-held controllers or head-gaze direction [26] for interaction and typing. While these offer benefits for locating and selecting targets, they are not suitable for interactions where hand tracking is posed as a valuable input technology [9]. Hand-tracking based studies in AR typing have predominantly focused on interaction, presenting gesture-based [9], pointing and selection mechanisms [40] and novel paradigms based on statistical models for improved typing reliability [8]. While these enable advancements in the field of AR/VR typing, configuration to their ergonomics, and specially the evaluation of position, orientation and location of virtual keyboards have not been previously explored.

Selecting character keys on virtual keyboards is often error-prone and inefficient [17, 22], as users may experience difficulties to locate small characters or have problems when performing locomotion tasks. Therefore, understanding the additional feedback methods that may aid users to improve their typing performance is an additional key challenge largely unexplored in the literature. The use of visual feedback such as glow effects have been explored with positive usability and performance results while interacting with virtual objects of different sizes in VR [11], mid-air typing in standard QWERTY layouts have received positive performance outcomes [42] while augmented visual effects have been explored in AR [45] with positive usability outcomes. The use of ray-casting methods have been proven helpful in AR typing while using handheld controllers [40], however to what extent ray-casting is still useful while using hand-tracking based approaches has not been explored.

This paper presents the first study comparing different AR keyboard positions and orientations alongside visual hand-based interaction feedback methods in AR character input scenarios. It contributes to the growing body of work in character input in immersive environments, where the delivery of productive and enjoyable input methods remains a challenge [7]. We present the results of a formal user study with 27 participants; evaluating 5 different keyboard locations (3 bound to HMD, and 2 bound to the non-dominant hand of the user) and 3 interaction feedback mechanisms (fingertip raycast, fingertip glow and both combined) as in Figure 1. This study reports on precision (time to completion, accuracy, Key Per Minute (KPM) and interaction metrics (After-Scenario Questionnaire (ASQ)). The paper is structured as follows; firstly we present a literature review of text input in immersive environments, methodology of the conducted study is then presented, followed by the precision and interaction metrics used. We then present the results analysis and conclusion to the work, detailing key aspects contributing to AR/VR typing.

# 2 Related Work - Text input in AR/VR

## 2.1 Physical keyboards and external devices

Physical keyboards have extensively been integrated in virtual environments, as a way of reducing the learning curve of new text input methods while keeping a familiar input mechanism [28]. Logitech released their own Software Developer Kit (SDK) as a consumer-ready solution to use physical keyboards in VR [4]. McGill et al. integrated
<figure><img src="image_3.png" /><figcaption>**Fig. 1.** Representation of the Meta 2 HMD with attached Leap Motion on top of the headset (a) and keyboard positions and visual feedback configurations (b-f). Keyboard Positions: Horizontal bound View Flat (VF) (b), Vertical Bound View Parallel (VP) (c) and HMD bound View Adaptive (VA) (d), Wrist bound Wrist Side (WS) (e) and Wrist Back (WB) (f). Interaction feedback visualisation conditions raycast and glow depicted together in illustrations (b) to (f).</figcaption></figure>

physical keyboards; augmenting the virtuality of typing tasks in VR and suggesting positive outcomes when enabling a partial view of hands and keyboard while interacting [24]; they followed up their study by integrating a realistic and co-located virtual representation of the keyboard in VR [25]. Grubert et al. also integrated standard physical keyboards in virtual environments, reporting on the effect of keyboard layouts and hand representations [13] on user performance [14]. Gupta et al. integrated physical keyboards while investigating different vibrotactile feedback conditions in typing tasks in VR [16], while Walker et al. investigated virtual assistants to assist in typing VR tasks when integrating physical keyboards [37]. Schneider et al. and Otter et al. further explored keyboard layouts while using touch-sensitive physical keyboards [29,34]. Pham and Stuerzlinger presented portable keyboard mounted on a hawker's tray for text entry while navigating or moving in VR [30].

Other devices such as haptic gloves [18, 39], custom-made wearables [15, 20] and hand-held controllers [3, 19, 44] have been studied for typing in AR/VR.

The majority of approaches involving physical keyboards have been developed for fully virtual environments, where the availability of consumer-ready VR equipment has resulted in in VR applications [6]. While physical keyboards can be connected to current AR HMDs such as the HoloLens, the use of traditional input devices such as mice and keyboard in AR environments are not suitable for outdoors or move-around/locomotion scenarios, as they require a surface to operate on [40] while proposing unique challenges, such as as the need for users to constantly switch from their virtual environment to the spatial layout and their surroundings to locate the physical keyboard [24]. Wearable and
external devices have proven to be useful for text entry in virtual environments while providing a haptic feeling. However, they often rely on ad-hoc hardware or keyboard layouts and interaction metaphors, limiting transferability and adoption.

## 2.2 Virtual keyboards in immersive environments

Grubert et al. compared the use of virtual and physical keyboards using a QWERTY layout [14] while Dudley et al. proposed a method to improve typing performance in AR environments [8]. Xu et al. investigated text entry mechanisms for virtual typing in AR environments. Results suggested that the combination of a controller and raycasting feedback methods outperformed other conditions [40].

Wang et al. presented PalmType a solution mapping the standard QWERTY keyboard layout to the palm as a text input method for smart-glasses [38]. This approach offered a novel QWERTY configuration and enabled passive feedback through the use of the body as a surface for binding AR typing interfaces.

Lee et al. developed a dynamic virtual keyboard layout for AR where the characters were positioned depending on their probability within the current input. However, they found dynamic layouts led to low accuracy and mediocre text input, so they proposed a 1-line solution keyboard that improved user performance [22].

Yu et al., Grossman et al. and Ogitani et al. explored different hand gestures and keyboard layouts to overcome spatial interaction limitations in smart-glasses [12, 27, 43].

Yu et al. presented a radial layout for text input using an standard game controller. This alternative presented characters in groups of 4 in alphabetical order, inside a circle partitioned in 7 slices [44]. Jiang and Weng also investigated a radial layout in combination with handheld devices as an alternative text input solution in VR environments [19].

Non-QWERTY alternatives have been explored predominantly to mitigate spatial and input restrictions in immersive environments. However, the main limitation of these approaches is that Users rarely invest time in learning new keyboard layouts [2, 22], and they are often not transferable limiting their wider adoption and suggesting that standard QWERTY layouts may be preferable.

## 2.3 Multi-modal approaches

Multi-modal approaches combining speech input, gaze tracking or head gaze rotation on their own or with handheld controllers or hand tracking methods have been explored in the literature. Speech input has been deemed to be the fastest medium for long text entry solutions [13]. Pick et al. presented a multi-modal interaction approach for text editing complementing speech input with point-and-click [31]. Other authors have used eye-tracking and head gaze in combination with dwell time, click interaction [33] and touch gestures [1] to reinforce gaze and head-gaze as interaction paradigm.

While these combination approaches have been deemed usable, no consideration have been given to the input of sensitive information or its usage in noisy and/or shared environments [13], while gaze interaction has previously shown to cause strain and motion sickness with prolonged text input tasks [33].
## 2.4 Visual feedback in typing interaction

Providing Interaction feedback while typing has a crucial impact on users' performance using HMDs [36]. Virtual keyboards in immersive environments are limited in size and FOV of the devices. Therefore, improvement in pointing performance and key aiming accuracy without interference is vital to the design of virtual keyboards [41]. Lee et al. used visual feedback as an additional support mechanism for text input in AR, using a colour changing ray-cast to showcase interaction state and width of the area [22].

The effect of hand representations on typing performance, workload and presence in VR was also explored [13, 21], while a minimalist representation of the users' fingertips did also enhance keyboard visibility [13]. Yang et al. used visual feedback in the shape of enhanced magnified raised keys to showcase when the fingers or participants were pointing to a particular key on the virtual keyboard in the VR environment [41].

Visual feedback have proven to be efficient in guiding user interaction while typing on immersive environments, supporting the use of novel interaction methods, increasing intuitiveness and influencing the feeling of presence, while reduced feedback could result in higher error rates [36]. Most of the approaches relied on ray-casting and pointing metaphors, and Xu et al. suggested the use of ray-casting as the preferred visual interaction feedback method for AR HMDs, linked to reduced motion sickness while using hand-held controllers [40]. However, to what extent this is maintained in freehand interaction have not been explored, while other aspects of visual enhancements, such as colour changes or other types of virtual highlighting, may also influence input performance [41].

# 3 Study Design

## 3.1 Apparatus

We built a custom experimental AR typing framework using Unity 2017.4.20f2 LTS, a Leap Motion sensor and the Leap Motion Core 4.4.0 SDK<fnref n="1" /> for hand tracking and the Meta2 headset for visualisation purposes as a head mounted display. C# was used as scripting language. The leap Motion sensor was mounted above the visor of the Meta2 headset, using a 3D printed structure tilted downwards following the shape of the device to facilitate hand tracking (Fig. 1a). We used the Meta 2 headset for visualisation due to its wider Field of View (FOV), however, we did not use built-in hand tracking capabilities due to reported limits with its tracking area and reliability [40].

Participants performed the test in a controlled environment under laboratory conditions. A non-cluttered room layout was used for all tests. The interactive space dimensions were 270 cm by 180 cm. The test room was lit by a 2700k (*warm white*) fluorescent with controlled external light source. A regular office chair (50 cm height) was used as participants were seated during the experiment. The chair was placed in the centre of the interaction space.

<footnote n="1">¹ https://developer.leapmotion.com/, (Last accessed 7th November 2020)</footnote>
## 3.2 Interaction

Previous typing studies have highlighted the difficulty of typing using multiple fingers in immersive environments, while proving that novice participants using a single digit per hand can perform better than users using all digits in multi-target selection tasks [7]. Therefore, we enabled dominant-hand interaction using the index finger, for direct interaction with the keyboard. Interaction was triggered upon collision detection of the fingertips with the keyboard position.

## 3.3 Conditions

Five different keyboard positions were evaluated in combination with three visual interaction feedback representations (5 × 3 = 15 unique conditions). Keyboard positions were not adjustable by participants.

**Keyboard positions** The keyboard position conditions are categorised into two primary conditions, viewpoint bound conditions (Fig. 1b-d) and non-dominant hand bound conditions (Fig. 1e-f). The virtual keyboard size was 26 cm width by 10 cm height. All keyboard positions were anchored relative to the user position and the position of the headset.

***Viewpoint bound conditions:*** The conditions showcased below were defined based on the position and orientation of the HMD. The keyboard was placed in a fixed plane 35 cm away in the Z axis and 10 cm below in the vertical plane from the camera (the HMD), changing its orientation as described in the conditions below. These are inspired by the tag-along User Interface (UI) paradigm common in current HMDs <fnref n="2" />.

- **View Flat (VF):** In this condition the keyboard was shown always horizontal (flat) in front of the user and anchored to the plane mentioned above. The position and orientation were calculated using the sensor information provided by the HMD. A representation of this condition is shown in Figure 1(b).

- **View Parallel (VP):** In this condition the keyboard was parallel to the HMD point of view. Thus was anchored parallel to the front facing position of the user, tilting its angle with head-gaze orientation and fixed to the plane mentioned earlier. A representation of this condition is shown in Figure 1(c).

- **View Adaptive (VA):** In this condition, the keyboard was anchored to the plane above but changed orientation between 2 tilting planes, one horizontal and one vertical, depending on HMD orientation. This transition between planes was performed using a threshold of 20 degrees below the horizon line of the HMD to avoid jittering, therefore removing the noise for involuntary head movements. A representation of this condition is shown in Figure 1(d).

<footnote n="2">² https://docs.microsoft.com/en-us/windows/mixed-reality/design/billboarding-and-tag-along, (Last accessed 14th January 2021)</footnote>
***Non-dominant hand bound conditions:*** The conditions showcased below were defined based on the position and orientation of the non-dominant hand.

- **Wrist Side (WS)** - In this condition the keyboard was displayed to the side of the users' wrist, floating in mid-air and 3 cm away from the wrist point. This was inspired by the Hand Menu UX pattern proposed by Microsoft in their Hololens 2³. The keyboard orientation was bound to the rotation of the wrist. A representation of these keyboard location is showcased in Figure 1(e).

- **Wrist Back (WB)** - The keyboard was displayed as an overlay between the elbow and wrist. This approach was inspired by previous research showcasing the benefits of passive feedback for interactions in virtual environments [7, 38]. A representation of this keyboard location is shown in Figure 1(f).

***Interaction feedback*** To support the interaction between the user's digit and the key to be pressed we evaluate two forms of visual feedback and guidance notably Raycast and Glow. These visual feedback methods are used for guiding the user and providing feedback about the key they are aiming at with their interaction.

***Raycast (R):*** This feedback method showed a continuous solid pointer connecting the users dominant hand index finger to the closest part of the surface of the keyboard. This feedback method is inspired by virtual pointing, one of the most common approaches used for object selection in immersive environments [32]. The raycast acts as a visual trajectory guidance method in locating the key for direct tap selection. A representation of this feedback condition is shown in Figure 2(a) and combined with the keyboard modes in 3(r), 3(n), 3(j), 3(b) and 3(f).

***Glow (G):*** In this condition the index fingertip in participants' dominant hand displayed a glow. Fingertip feedback has proven to enhance keyboard visibility [13] and has recently been adopted by Microsoft UX guidelines for direct object manipulation using their Collidable Fingertip paradigm<fnref n="4" />. This glow represented the contact point that will be used for interaction with the keyboard, changing its size and intensity with proximity to the target. This glow effect is inspired by previous work [11], where authors evaluated glow effects as an interaction trigger in VR. A representation of this feedback condition is shown in Figure 2(b) and combined with the keyboard modes in 3(s), 3(o), 3(k), 3(c) and 3(g).

***Both combined (B):*** This condition combined both the raycast and glow feedback to display both concurrently. A representation of this feedback condition is shown in Figure 2(c) and combined with the keyboard modes in 3(t), 3(p), 3(l), 3(d) and 3(h).

<footnote n="3">³ https://docs.microsoft.com/en-us/windows/mixed-reality/design/hand-menu, (Last accessed 14th January 2020)</footnote>

<footnote n="4">⁴ https://docs.microsoft.com/en-us/windows/mixed-reality/design/direct-manipulationinteraction, (Last accessed 7th November 2020)</footnote>
<figure><img src="image_4.png" /><figcaption>**Fig. 2.** Interaction feedback modes. Raycast depicting a continuous guidance ray to the keyboard (a), Glow depicting the tip of the dominant hand index finger (b) and both combined (c). Black hand represents the AR hand segmentation alpha mask for the occlusion handling of the interaction hand, grey region is rendered to the real environment on the AR HMD.</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>**Fig. 3.** Interaction modes for the keyboard and the visual feedback in the test. Representations of the five keyboard positions with the corresponding representations of the feedback modes for each keyboard representation. VF stands for View Flat, VA for View Adaptive, VP for View Parallel, WS for Wrist Side and WB for Wrist Back keyboard placement conditions. Black hand represents the AR hand segmentation mask for the occlusion handling of the interaction hand, grey region corresponds to the real environment of the user. The virtual keyboard and feedback mechanisms were the only elements rendered in the device.</figcaption></figure>
## 3.4 Task

We proposed these tasks under the scenario of security/sensitive character input (as in passwords), where hand interaction will be preferred over speech commands for privacy. The input task was defined to involve the use of special characters, numbers and capital letters in character strings, in compliance with standardise guidelines for secure passwords. Therefore, participants were asked to input a randomised 7-character sequence containing a) 4 letters (2 of them uppercase) b) 2 numbers and c) 1 special character (e.g. G2+hDf8). Sequences were presented above the keyboard in the UI layout and they maintained the same level of complexity for every condition under study, each participant completed a total of 15 unique sequences. Lower case L and upper case I were removed from all sequences for clarity.

## 3.5 Participants

27 right-handed participants (1 non-binary, 6 female, 20 male) from a population of university students and staff members were recruited to take part in this study. Participants' mean age was 24.3 (SD: 4.78).

All participants performed the task described in section 3.4 under the keyboard placement and feedback conditions explained in section 3.3. Participants completed a standardised consent form and were not compensated. Visual acuity of participants was measured using a Snellen chart, each participant was also required to pass an Ishihara test to exclude for colour blindness. Participants with colour blindness and/or visual acuity of < 0.80 (where 20/20 is 1.0) were not included in this study.

Participants were asked to self-assess their level of experience with AR and VR systems, with 5 participants reporting to have an average level of experience and the remaining 22 reported being novice to immersive technologies. None of the participants had any substantial previous experience using an AR HMD or hand tracking devices.

## 3.6 Protocol

A within participants test protocol design was used. All participants tested the 5 keyboard positions and 3 interaction methods, inputting a total of 15 string sequences. Participants were only allowed to proceed to the next sequence when the current input was correct. The overall duration of the experiment ranged between 30 and 45 minutes, including pre-test and post-test questionnaires.

**Pre-Test** Prior to the study, participants were given a written consent form, where the test protocol was described. Additionally, participants completed a pre-test questionnaire enquiring about their background level of experience with immersive systems, recognition sensors and the use of HMDs.

**Calibration** Before each test, the test coordinator helped participants to fit the HMD in the most suitable and comfortable way to ensure successful hand tracking from the Leap motion. Once the system was calibrated participants were asked to confirm that the characters on display were clearly legible.
**Training** Once participants were comfortable with the device and the hand tracking, recognition and interaction system, they were trained with a specific task-related scenario. This consisted of 3 training tasks introducing different levels of typing complexity: 1) participants were asked to input a lowercase 3 character sequence, 2) participants were introduced to the SHIFT key for uppercase and special characters, 3) participants were asked to use the arrow keys and delete button to edit their inputs. Each training sequence was a predefined string with 3 characters. This task training was the same for every participant and they were trained in a representative version of every keyboard position.

**Test** Once participants were comfortable with the interaction conditions, we presented the main experimental task. Participants were asked to complete the task as accurately as possible in the shortest amount of time. Tasks reported in 3.4 for every condition reported in 3.3 were loaded in counterbalanced random order.

**Post-Test** After each of the keyboard-location conditions were completed, participants were asked to fill the After Scenario Questionnaire (ASQ). Once all 5 keyboard location conditions were completed and all the interaction tests were finished, participants were asked to fill a post-test questionnaire asking about their overall experience with the system and keyboard location and visual feedback preferences.

**3.7 Metrics**

**Completion time** Completion time was defined as the time it took to complete the string input task per condition under study. It was measured as the time in seconds from the start of every condition until the participant successfully input the character string.

**Accuracy** Accuracy was calculated as correct key presses / total key presses per task (task defined as the combination of keyboard location and interaction feedback).

Each key-press was labelled as correct or incorrect at run-time based on the considerations listed below:

**Labelled as correct:** a) After a key-press, if the input matches the character sequence up to that point. b) If the key-press is BACKSPACE, if there was a mistake behind the cursor. c) If the key-press is CLEAR, if there was any mistake in the text. d) LEFT if there is an error more than one character to the left of the cursor. e) RIGHT if there is an error to the right of the cursor, or there are no errors and the cursor is in the middle of the input. f) RETURN if the input matches the character sequence.

**Labelled as incorrect:** a) If the key-press is a letter/special character and the input does not match the character sequence. b) If the key-press is SHIFT and it would result in the next input character not matching the next sequence character.
**Key per minute (KPM)** Entry rate was measured in keys per minute (KPM). KPM was calculated as the number of key-press in a minute calculated based on the ratio of input obtained for every task as in $\frac{\text{key input} \times 60 \text{ seconds}}{\text{time in seconds}}$.

**After Scenario Questionnaire (ASQ)** This questionnaire was used to assess participants' satisfaction or frustration after the completion of each task per condition. This three-item questionnaire address ease of task completion, time to complete a task, and adequacy of the support information [23]. It gives a value from 1 *strongly agree* to 7 *strongly disagree*; being 1 the ideal.

## 3.8 Statistical Analysis

The Shapiro-Wilk [35] normality test found the data to be not normally distributed. We tested for significance between the conditions and the metrics described using a non parametric Friedman test [10]. 95% Confidence Intervals (CI) and pair-wise Effect Sizes (ES) are reported.

# 4 Results

A comprehensive analysis of completion time, accuracy and KPM with Effect Sizes (ES) and 95% Confidence Intervals (CIs) per keyboard position and interaction feedback condition is presented in Tables 2 and 3 respectively. A comprehensive statistical analysis of ASQ responses is presented in Table 1.

## 4.1 Completion Time

Completion time results by keyboard position condition are shown in Figure 4(a). Completion times ranged from 62.35 sec (SD = 45.03 sec) for Wrist-Back Glow condition to 25.70 sec (SD = 10.04 sec) for the View-Parallel Glow condition.

**By Keyboard Position** Statistically significant differences were found for the different interaction feedback modes for Wrist-Side keyboard position, with medium ES between a) Glow and Both combined b) Raycast and Both combined. No statistically significant differences were found for the remaining keyboard conditions comparing interaction feedback modes for each position (i.e. *raycast* vs. *glow* and *glow* vs. *both* for *View Parallel*). An in-depth analysis of these results is presented in Table 2.

### By Feedback Mode

- **Raycast:** Statistically significant differences were found between keyboard locations for the raycast feedback condition with large ES shown between a Wrist-Back and View-Parallel.

- **Glow:** Statistically significant differences were found between keyboard locations for glow feedback mode. Large ES were shown between a) Wrist-Back and View-Parallel, b) Wrist-Back and View-Adaptive c) View-Parallel and View-Flat.
- **Both:** Statistically significant differences were found between keyboard locations for the combined feedback mode. While medium ES where shown as in Table 3 no large ES were found.

<figure><img src="image_6.png" /><figcaption>**Fig. 4.** Completion time 4(a), accuracy 4(b) and KPM 4(c) per keyboard position and feedback condition displayed in seconds with white triangles showcasing mean value, where VF stands for View Flat, VA for View Adaptive, VP for View Parallel, WS for Wrist Side and WB for Wrist Back keyboard placement conditions, with R standing for Raycast feedback condition, G for Glow and B for both feedback conditions.</figcaption></figure>

## 4.2 Accuracy

Accuracy results per keyboard location and feedback mode are displayed on Figure 4(b). Average accuracy levels were high for all conditions, ranging from 0.93 (SD = 0.07) for View-Parallel Both condition to 0.85 (SD = 0.12) for the Wrist Back Glow condition.

No statistically significant differences were found between the visual feedback modes when comparing them for each keyboard placement position. The comparison between different keyboard positions when comparing them by feedback mode did not show statistically significant differences either. This suggested that the interaction feedback modes and keyboard positions under study did not have statistically significant effects on input accuracy.

## 4.3 Key per Minute (KPM)

Key per Minute (KPM) values are shown on Figure 4(c). Average values ranged from 36.29 kpm (SD = 9.47 kpm) for the View-Parallel Glow condition to 24.7 kpm (SD
= 10.06 kpm) and 24.29 kpm (SD = 9.45 kpm) for the View-Flat Raycast and Glow conditions.

**By Keyboard Position** Statistically significant differences were found again for Wrist-Back keyboard condition, with medium ES showing between a) Glow and both feedback combined. No statistically significant differences were found for the remaining keyboard conditions comparing interaction feedback modes for each position. An in-depth analysis of this is presented in Table 2.

**By Feedback Mode**

- **Raycast:** Statistically significant differences were found between keyboard locations for raycast feedback. Large ES were shown between a) Wrist-Back and View-Parallel b) Wrist-Back and View-Adaptive c) View-Parallel and View-Flat.

- **Glow:** Statistically significant differences were found between keyboard locations for glow feedback. Large ES were found for a) Wrist-Side and View-Parallel b) Wrist-Back and View-Parallel c) View-Parallel and View-Flat.

- **Both:** Statistically significant differences were found between keyboard locations for combined feedback. While medium ES where shown as in Table 3 no large ES were found.

**4.4 After-Scenario Questionnaire (ASQ)**

Statistically significant differences were found between keyboard location conditions for ASQ questionnaire, as in Table 1. Overall scores per condition are showcased in Figure 5. Mean scores ranged from 1.88 (SD = 1.03) for *View Parallel* condition to 3.48 (SD = 1.59) for the *Wrist Back* condition, with Wrist Side scoring 2.41 (SD = 1.16), View Adaptive 2.14 (SD = 1.26) and View Flat 2.96 (SD = 1.27). Large ES were found between a) Wrist-Back and View-Parallel b) Wrist-Back and View-Adaptive.

**4.5 Preferences**

Participants were asked to rank keyboard locations and interaction feedback modes in preference order (from 1 to 5 for keyboard locations and 1 to 3 for feedback conditions). No tracking issues were reported by participants and results are presented in Figures 6(a) and 6(b).

- **Keyboard position:** 13 participants chose "View-Parallel" as their preferred keyboard position, while 15 participants declared that "Wrist-Back" was their least preferred position for text input interaction. This results are in alignment with the performance metrics presented earlier, where View Parallel had one of the shortest completion times and highest KPM and accuracy values. Results for all the location conditions are presented in Figure 6(a).
<table><caption>**Table 2.** Completion time, accuracy and Key per Minute (KPM) statistics, displaying effect sizes (|ES|) and 95% confidence intervals (CI) where G stands for Glow feedback mode, R for Raycast feedback mode and B for Both feedback combined for all keyboard location conditions.</caption><thead><tr><th></th><th></th><th>Wrist Side</th><th>Wrist Back</th><th>View Adaptive</th><th>View Parallel</th><th>View Flat</th></tr></thead><tbody><tr><td rowspan="3">Completion Time</td><td>p</td><td>(Stat = 8.22, p = 0.01)*</td><td>(Stat = 1.85, p = 0.39)</td><td>(Stat = 1.18, p = 0.55)</td><td>(Stat = 0.29, p = 0.86)</td><td>(Stat = 1.55, p = 0.45)</td></tr><tr><td>|ES|</td><td>B vs. G = 0.51<br>B vs. R = 0.61<br>R vs. G = 0.01</td><td>B vs. G = 0.25<br>B vs. R = 0.19<br>R vs. G = 0.05</td><td>B vs. G = 0.19<br>B vs. R = 0.03<br>R vs. G = 0.14</td><td>B vs. G = 0.04<br>B vs. R = 0.19<br>R vs. G = 0.23</td><td>B vs. G = 0.09<br>B vs. R = 0.26<br>R vs. G = 0.30</td></tr><tr><td>95% CI</td><td>B R 0 20 40 60 80 100 Time [sec]</td><td>B R 0 20 40 60 80 100 Time [sec]</td><td>B R 0 20 40 60 80 100 Time [sec]</td><td>B R 0 20 40 60 80 100 Time [sec]</td><td>B R 0 20 40 60 80 100 Time [sec]</td></tr><tr><td rowspan="3">Accuracy</td><td>p</td><td>(Stat = 0.43, p = 0.80)</td><td>(Stat = 0.42, p = 0.81)</td><td>(Stat = 0.64, p = 0.73)</td><td>(Stat = 0.92, p = 0.63)</td><td>(Stat = 2.58, p = 0.27)</td></tr><tr><td>|ES|</td><td>B vs. G = 0.17<br>B vs. R = 0.18<br>R vs. G = 0.03</td><td>B vs. G = 0.25<br>B vs. R = 0.02<br>R vs. G = 0.23</td><td>B vs. G = 0.03<br>B vs. R = 0.34<br>R vs. G = 0.31</td><td>B vs. G = 0.31<br>B vs. R = 0.37<br>R vs. G = 0.07</td><td>B vs. G = 0.1<br>B vs. R = 0.2<br>R vs. G = 0.32</td></tr><tr><td>95% CI</td><td>B R 0.0 0.2 0.4 0.6 0.8 1.0 Accuracy</td><td>B R 0.0 0.2 0.4 0.6 0.8 1.0 Accuracy</td><td>B R 0.0 0.2 0.4 0.6 0.8 1.0 Accuracy</td><td>B R 0.0 0.2 0.4 0.6 0.8 1.0 Accuracy</td><td>B R 0.0 0.2 0.4 0.6 0.8 1.0 Accuracy</td></tr><tr><td rowspan="3">Key per Minute (KPM)</td><td>p</td><td>(Stat = 6.66, p = 0.03)*</td><td>(Stat = 2.28, p = 0.32)</td><td>(Stat = 0.47, p = 0.78)</td><td>(Stat = 1.41, p = 0.49)</td><td>(Stat = 1.27, p = 0.52)</td></tr><tr><td>|ES|</td><td>B vs. G = 0.54<br>B vs. R = 0.48<br>R vs. G = 0.01</td><td>B vs. G = 0.26<br>B vs. R = 0.45<br>R vs. G = 0.20</td><td>B vs. G = 0.09<br>B vs. R = 0.03<br>R vs. G = 0.06</td><td>B vs. G = 0.22<br>B vs. R = 0.06<br>R vs. G = 0.31</td><td>B vs. G = 0.22<br>B vs. R = 0.17<br>R vs. G = 0.04</td></tr><tr><td>95% CI</td><td>B R 0 10 20 30 40 50 [KPM]</td><td>B R 0 10 20 30 40 50 [KPM]</td><td>B R 0 10 20 30 40 50 [KPM]</td><td>B R 0 10 20 30 40 50 [KPM]</td><td>B R 0 10 20 30 40 50 [KPM]</td></tr></tbody></table>

to privacy concerns. These tasks are common in HMD interaction, specially when connecting the device to a WiFi network or inputting account credentials.

Most current text entry solutions present the virtual keyboard location anchored and fixed in 3D space, with current text input methods for immersive environments not allowing the user to change the position or size of keyboard representations [36]. This study presented a unique approach to varying the position and orientation of the virtual keyboard based on participants position and orientation.

The results showcased a preference for the *View Parallel* condition for completion time, ASQ scores and KPM metrics. Key input metrics for this condition were in alignment with those reported in the literature and deemed as tolerable [8]. We envisage participants performance will improve with experience and practice. Therefore, we presented these results as an indication of achievability under the circumstances outlined.
<table><caption>**Table 3.** Completion time, accuracy and key per minute (KPM) statistics, showcasing effect sizes (|ES|) where WS stands for Wrist Back keyboard placement, WB for Wrist Back keyboard placement, VP for View Parallel keyboard placement, VA for View Adaptive keyboard placement and VF for View Flat keyboard placement for all feedback conditions.</caption><thead><tr><th></th><th></th><th>Raycast</th><th>Glow</th><th>Both</th></tr></thead><tbody><tr><td rowspan="6">Completion Time</td><td>p</td><td>(Stat = 16.50,<br>p = 0.02)*</td><td>(Stat = 24.5,<br>p &lt; 0.01)*</td><td>(Stat = 9.27,<br>p = 0.05)*</td></tr><tr><td>|ES|</td><td>WS vs. WB = 0.55<br>WS vs. VP = 0.49<br>WS vs. VA = 0.38<br>WS vs. VF = 0.26<br>WB vs. VP = 0.84</td><td>WS vs. WB = 0.63<br>WS vs. VP = 0.59<br>WS vs. VA = 0.20<br>WS vs. VF = 0.52<br>WB vs. VP = 0.90</td><td>WS vs. WB = 0.70<br>WS vs. VP = 0.09<br>WS vs. VA = 0.27<br>WS vs. VF = 0.60<br>WB vs. VP = 0.74</td></tr><tr><td>95% CI</td><td>WB<br>WS<br>VA<br>VF<br>0 20 40 60 80 100<br>Time [sec]</td><td>WB<br>WS<br>VA<br>VF<br>0 20 40 60 80 100<br>Time [sec]</td><td>WB<br>WS<br>VA<br>VF<br>0 20 40 60 80 100<br>Time [sec]</td></tr><tr><td rowspan="6">Accuracy</td><td>p</td><td>(Stat = 1.63,<br>p = 0.80)</td><td>(Stat = 5.82,<br>p = 0.2)</td><td>(Stat = 5.80,<br>p = 0.21)</td></tr><tr><td>|ES|</td><td>WS vs. WB = 0.18<br>WS vs. VP = 0.08<br>WS vs. VA = 0.12<br>WS vs. VF = 0.21<br>WB vs. VP = 0.11</td><td>WS vs. WB = 0.51<br>WS vs. VP = 0.05<br>WS vs. VA = 0.19<br>WS vs. VF = 0.58<br>WB vs. VP = 0.46</td><td>WS vs. WB = 0.45<br>WS vs. VP = 0.08<br>WS vs. VA = 0.39<br>WS vs. VF = 0.57<br>WB vs. VP = 0.56</td></tr><tr><td>95% CI</td><td>WB<br>WS<br>VA<br>VF<br>0.0 0.2 0.4 0.6 0.8 1.0<br>Accuracy</td><td>WB<br>WS<br>VA<br>VF<br>0.0 0.2 0.4 0.6 0.8 1.0<br>Accuracy</td><td>WB<br>WS<br>VA<br>VF<br>0.0 0.2 0.4 0.6 0.8 1.0<br>Accuracy</td></tr><tr><td rowspan="6">Key per Minute (KPM)</td><td>p</td><td>(Stat = 25.44,<br>p &lt; 0.01)*</td><td>(Stat = 35.82,<br>p &lt; 0.01)*</td><td>(Stat = 10.31,<br>p = 0.03)*</td></tr><tr><td>|ES|</td><td>WS vs. WB = 0.51<br>WS vs. VP = 0.49<br>WS vs. VA = 0.26<br>WS vs. VF = 0.26<br>WB vs. VP = 0.93</td><td>WS vs. WB = 0.32<br>WS vs. VP = 0.87<br>WS vs. VA = 0.25<br>WS vs. VF = 0.33<br>WB vs. VP = 0.95</td><td>WS vs. WB = 0.48<br>WS vs. VP = 0.05<br>WS vs. VA = 0.24<br>WS vs. VF = 0.60<br>WB vs. VP = 0.52</td></tr><tr><td>95% CI</td><td>WB<br>WS<br>VA<br>VF<br>0 10 20 30 40 50<br>KPM</td><td>WB<br>WS<br>VA<br>VF<br>0 10 20 30 40 50<br>KPM</td><td>WB<br>WS<br>VA<br>VF<br>0 10 20 30 40 50<br>KPM</td></tr></tbody></table>
While previous studies have shown a preference for tilted keyboards in VR [5, 41], they have mostly evaluated keyboard positions anchored to spatial surroundings. We proposed these alternatives to spatially fixed keyboards to support on demand access to typing capabilities, specially for inputting confidential information.

Limitations of current HMD devices and tracking solutions have showcased limits in the use of horizontal keyboards. Primarily due to tracking inaccuracies while maintaining a low head posture which may end in increased neck pressure [41] and discomfort for users. This is in alignment with the results presented, where the *View Flat* condition received poor ASQ scores. Therefore, the use of horizontal keyboards, akin to current interaction with physical keyboards, may require further technical support [41] for them to be adopted. While passive feedback has been successfully explored in immersive environments before [38], we did not find the same results. While no tracking issues were reported during the study, this could be due to current limitation of the display and hand tracking technologies and may need to be further explored with improved tracking capabilities in the future.

We paired keyboard location with the evaluation of visual interaction feedback to further assess visual guidance and its effect on perceived usability and character input performance. While no statistically significant differences were found, ray-cast and fingertip feedback have been used widely in AR/VR environments and our suggestion would be for it to be considered and implemented as a guiding technique. Previous literature suggested that reduced visual feedback could result in decreased performance [36], therefore we suggest supportive visual feedback to be considered when enabling text input.

Overall, the main findings of the study related to short character input, akin to password and confidential information input, in AR environments were as follows:

- Keyboard locations bounded to the user and following the same orientation of the viewpoint of the HMD were preferred for interaction (i.e. *always in front keyboards*).

- While there was no clear preference for a visual feedback mode, our suggestion would be to continue to use ray-cast and fingertip visual feedback for guiding typing tasks in immersive environments.

- Multi-modal interaction (i.e. speech and gaze) and haptic feedback should be considered, specially for longer non-confidential typing tasks.

## 5.1 Limitations

Freehand input has been deemed as the most realistic text entry method for immersive environments [36]. This technique relies on hand tracking capabilities, therefore we employed the Leap Motion sensor mounted on the front of the AR HMD. While this approach has been previously used for text entry studies [41], it is worth highlighting that the tracking limitations of this device may negatively impact performance and preference results. However, the Leap Motion was chosen as it is the current most affordable and consumer available method for finger input. Future developments in hand tracking technology, specifically related to robustness and accuracy, may influence the results presented here. Furthermore, while external and high-quality motion tracking solutions could be employed to track the user, the current system presents a deployable
solution that can easily be adopted and utilised in wider studies with comparable HMD setups, as motion capture solutions are not portable or mobile [21].

## 5.2 Future work

Our evaluation was conducted in a laboratory environment, under controlled conditions. This supports repeatability and transferability of the findings, however, future work should consider more realistic locomotion situations and environments i.e. outdoors, on the move, while commuting, etc. This will expand the finding from this work into text input interaction paradigms and keyboard locations that enable the user to fully move and explore the AR environment. These situations may also require keyboards and interactive elements to be bound to the user and not the environment, thus leveraging the learned keyboard interaction behaviour and exploiting the additional dimensions of the space available in AR [8]. This future research can be informed by the results of the current experiment and be extrapolated to wider interaction scenarios such as user interface widgets and buttons.

We solely evaluated visual feedback for typing guidance, future work should consider the role of audio feedback and haptics as further guidance support for typing tasks in AR.

Although the presented solution may be suitable for short character input sequences or sensitive information such as passwords, it may be worth considering combination approaches using speech input for longer text editing or writing tasks. It is worth exploring these to support text entry techniques that are more involved [40].

# 6 Conclusion

We have conducted a study comparing five different keyboard locations and three visual feedback modes in a controlled AR environment for text input. We followed a within participants study design and we reported on completion time, KPM, accuracy and ASQ questionnaire metrics. Our results suggest that *View Parallel* condition outperformed all keyboard locations while the visual feedback used did not have a statistically significant effect.

Our findings have some interesting implications for the design and implementation of text input tasks in AR. Overall, the results of this study can guide the design of typing on immersive environments by applying the keyboard location and input feedback considerations presented here, specially for tasks that require the input of sensitive content of login credentials.

## References

1. Ahn, S., Lee, G.: Gaze-assisted typing for smart glasses. In: Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology. pp. 857–869 (2019)

2. Bi, X., Smith, B.A., Zhai, S.: Quasi-qwerty soft keyboard optimization. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. pp. 283–286 (2010)
3. Boletsis, C., Kongsvik, S.: Text input in virtual reality: A preliminary evaluation of the drum-like vr keyboard. Technologies 7(2), 31 (2019)

4. Bovet, S., Kehoe, A., Crowley, K., Curran, N., Gutierrez, M., Meisser, M., Sullivan, D.O., Rouvinez, T.: Using traditional keyboards in vr: Steamvr developer kit and pilot game user study. In: 2018 IEEE Games, Entertainment, Media Conference (GEM). pp. 1–9. IEEE (2018)

5. Dang, N.T., Mestre, D.: Effects of menu orientation on pointing behavior in virtual environments. In: International Conference on Virtual and Mixed Reality. pp. 144–153. Springer (2011)

6. Dube, T.J., Arif, A.S.: Text entry in virtual reality: A comprehensive review of the literature. In: International Conference on Human-Computer Interaction. pp. 419–437. Springer (2019)

7. Dudley, J., Benko, H., Wigdor, D., Kristensson, P.O.: Performance envelopes of virtual keyboard text input strategies in virtual reality. In: 2019 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). pp. 289–300. IEEE (2019)

8. Dudley, J.J., Vertanen, K., Kristensson, P.O.: Fast and precise touch-based text entry for head-mounted augmented reality with variable occlusion. ACM Transactions on Computer-Human Interaction (TOCHI) 25(6), 1–40 (2018)

9. Fashimpaur, J., Kin, K., Longest, M.: Pinchtype: Text entry for virtual and augmented reality using comfortable thumb to fingertip pinches. In: Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems Extended Abstracts. pp. 1–7 (2020)

10. Friedman, M.: A comparison of alternative tests of significance for the problem of m rankings. The Annals of Mathematical Statistics 11(1), 86–92 (1940)

11. Frutos-Pascual, M., Harrison, J.M., Creed, C., Williams, I.: Evaluation of ultrasound haptics as a supplementary feedback cue for grasping in virtual environments. In: 2019 International Conference on Multimodal Interaction. pp. 310–318 (2019)

12. Grossman, T., Chen, X.A., Fitzmaurice, G.: Typing on glasses: Adapting text entry to smart eyewear. In: Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services. pp. 144–152 (2015)

13. Grubert, J., Witzani, L., Ofek, E., Pahud, M., Kranz, M., Kristensson, P.O.: Effects of hand representations for typing in virtual reality. In: 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). pp. 151–158. IEEE (2018)

14. Grubert, J., Witzani, L., Ofek, E., Pahud, M., Kranz, M., Kristensson, P.O.: Text entry in immersive head-mounted display-based virtual reality using standard keyboards. In: 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). pp. 159–166. IEEE (2018)

15. Gupta, A., Ji, C., Yeo, H.S., Quigley, A., Vogel, D.: Rotoswype: Word-gesture typing using a ring. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. pp. 1–12 (2019)

16. Gupta, A., Samad, M., Kin, K., Kristensson, P.O., Benko, H.: Investigating remote tactile feedback for mid-air text-entry in virtual reality. In: 2020 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). pp. 350–360. IEEE (2020)

17. Hincapié-Ramos, J.D., Guo, X., Irani, P.: The consumed endurance workbench: a tool to assess arm fatigue during mid-air interactions. In: Proceedings of the 2014 companion publication on Designing interactive systems, pp. 109–112 (2014)

18. Hsieh, Y.T., Jylhä, A., Orso, V., Gamberini, L., Jacucci, G.: Designing a willing-to-use-in-public hand gestural interaction technique for smart glasses. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. pp. 4203–4215 (2016)

19. Jiang, H., Weng, D.: Hipad: Text entry for head-mounted displays using circular touchpad

20. Kim, J., Delamare, W., Irani, P.: Thumbtext: text entry for wearable devices using a miniature ring. In: Proceedings of Graphics Interface. pp. 18–25 (2018)

21. Knierim, P., Schwind, V., Feit, A.M., Nieuwenhuizen, F., Henze, N.: Physical keyboards in virtual reality: Analysis of typing performance and effects of avatar hands. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. pp. 1–9 (2018)
22. Lee, L.H., Braud, T., Lam, K.Y., Yau, Y.P., Hui, P.: From seen to unseen: Designing keyboard-less interfaces for text entry on the constrained screen real estate of augmented reality headsets. Pervasive and Mobile Computing p. 101148 (2020)

23. Lewis, J.R.: Ibm computer usability satisfaction questionnaires: psychometric evaluation and instructions for use. International Journal of Human-Computer Interaction **7**(1), 57–78 (1995)

24. McGill, M., Boland, D., Murray-Smith, R., Brewster, S.: A dose of reality: Overcoming usability challenges in vr head-mounted displays. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. pp. 2143–2152 (2015)

25. Mcgill, M., Kehoe, A., Freeman, E., Brewster, S.: Expanding the bounds of seated virtual workspaces. ACM Transactions on Computer-Human Interaction (TOCHI) **27**(3), 1–40 (2020)

26. Menzner, T., Otte, A., Gesslein, T., Grubert, J., Gagel, P., Schneider, D.: A capacitive-sensing physical keyboard for vr text entry. In: 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). pp. 1080–1081. IEEE (2019)

27. Ogitani, T., Arahori, Y., Shinyama, Y., Gondow, K.: Space saving text input method for head mounted display with virtual 12-key keyboard. In: 2018 IEEE 32nd International Conference on Advanced Information Networking and Applications (AINA). pp. 342–349. IEEE (2018)

28. Otte, A., Menzner, T., Gesslein, T., Gagel, P., Schneider, D., Grubert, J.: Towards utilizing touch-sensitive physical keyboards for text entry in virtual reality. In: 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). pp. 1729–1732. IEEE (2019)

29. Otte, A., Schneider, D., Menzner, T., Gesslein, T., Gagel, P., Grubert, J.: Evaluating text entry in virtual reality using a touch-sensitive physical keyboard. In: 2019 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct). pp. 387–392. IEEE (2019)

30. Pham, D.M., Stuerzlinger, W.: Hawkey: Efficient and versatile text entry for virtual reality. In: 25th ACM Symposium on Virtual Reality Software and Technology. pp. 1–11 (2019)

31. Pick, S., Puika, A.S., Kuhlen, T.W.: Swifter: Design and evaluation of a speech-based text input metaphor for immersive virtual environments. In: 2016 IEEE Symposium on 3D User Interfaces (3DUI). pp. 109–112. IEEE (2016)

32. Pietroszek, K.: Raycasting in Virtual Reality, pp. 1–3. Springer International Publishing, Cham (2018). https://doi.org/10.1007/978-3-319-08234-9_180-1, https://doi.org/10.1007/978-3-319-08234-9_180-1

33. Rajanna, V., Hansen, J.P.: Gaze typing in virtual reality: impact of keyboard design, selection method, and motion. In: Proceedings of the 2018 ACM Symposium on Eye Tracking Research & Applications. pp. 1–10 (2018)

34. Schneider, D., Otte, A., Gesslein, T., Gagel, P., Kuth, B., Damlakhi, M.S., Dietz, O., Ofek, E., Pahud, M., Kristensson, P.O., et al.: Reconfiguration: Reconfiguring physical keyboards in virtual reality. IEEE transactions on visualization and computer graphics **25**(11), 3190–3201 (2019)

35. Shapiro, S.S., Wilk, M.B.: An analysis of variance test for normality (complete samples). Biometrika **52**(3/4), 591–611 (1965)

36. Speicher, M., Feit, A.M., Ziegler, P., Krüger, A.: Selection-based text entry in virtual reality. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. pp. 1–13 (2018)

37. Walker, J., Li, B., Vertanen, K., Kuhl, S.: Efficient typing on a visually occluded physical keyboard. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. pp. 5457–5461 (2017)

38. Wang, C.Y., Chu, W.C., Chiu, P.T., Hsiu, M.C., Chiang, Y.H., Chen, M.Y.: Palmtime: Using palms as keyboards for smart glasses. In: Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services. pp. 153–160 (2015)

39. Wu, C.M., Hsu, C.W., Lee, T.K., Smith, S.: A virtual reality keyboard with realistic haptic feedback in a fully immersive virtual environment. Virtual Reality **21**(1), 19–29 (2017)
40. Xu, W., Liang, H.N., He, A., Wang, Z.: Pointing and selection methods for text entry in augmented reality head mounted displays. In: 2019 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). pp. 279–288. IEEE (2019)

41. Yang, Z., Chen, C., Lin, Y., Wang, D., Li, H., Xu, W.: Effect of spatial enhancement technology on input through the keyboard in virtual reality environment. Applied ergonomics **78**, 164–175 (2019)

42. Yi, X., Yu, C., Zhang, M., Gao, S., Sun, K., Shi, Y.: Atk: Enabling ten-finger freehand typing in air based on 3d hand tracking data. In: Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology. pp. 539–548 (2015)

43. Yu, C., Sun, K., Zhong, M., Li, X., Zhao, P., Shi, Y.: One-dimensional handwriting: Inputting letters and words on smart glasses. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. pp. 71–82 (2016)

44. Yu, D., Fan, K., Zhang, H., Monteiro, D., Xu, W., Liang, H.N.: Pizzatext: text entry for virtual reality systems using dual thumbsticks. IEEE Transactions on Visualization and Computer Graphics **24**(11), 2927–2935 (2018)

45. Yumura, T., Nakamura, S.: Augmented typing: Augmentation of keyboard typing experience by adding visual and sound effects. In: Asian CHI Symposium 2019 (2019)