# The Impact of Visual and Haptic Feedback on Keyboard Typing in Immersive Virtual Environments

Amine Chellali, Lucas Herfort, Guillaume Loup, Marie-Hélène Ferrer

► To cite this version:

Amine Chellali, Lucas Herfort, Guillaume Loup, Marie-Hélène Ferrer. The Impact of Visual and Haptic Feedback on Keyboard Typing in Immersive Virtual Environments. IEEE Transactions on Visualization and Computer Graphics, 2025, 31 (5), pp.2633–2642. 10.1109/TVCG.2025.3549555. hal-04955458

HAL Id: hal-04955458

https://hal.science/hal-04955458v1

Submitted on 13 Mar 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# The Impact of Visual and Haptic Feedback on Keyboard Typing in Immersive Virtual Environments

Amine Chellali ID, Lucas Herfort ID, Guillaume Loup ID, and Marie-Hélène Ferrer ID

<figure><img src="image_2.png" /><figcaption>Fig. 1: Conditions studied in the experiment. Haptic feedback conditions: (Left) Physical Keyboard (Desktop), (center-down) Flat Surface (Surface), (Right-down) virtual keyboard with no haptic feedback (Mid-air). Visual feedback conditions: (center-top) Hover feedback (on or off), (right-top) Keypress feedback (on or off).</figcaption></figure>

**Abstract**—Typing with a keyboard is a common task in content production in the workplace. For simulation purposes in VR environments, it is important for users to perform this task accurately, with minimal performance loss, and without distraction. One common approach is using mid-air typing on virtual keyboards. However, this method presents challenges, particularly due to the lack of haptic feedback and spatial awareness. Various solutions have been suggested in the literature to address these challenges, but several design factors that influence performance, behavior, and user experience still need to be explored. This paper investigates the effects of two types of visual feedback (hover and keypress) and three passive haptic feedback conditions (physical keyboard, physical surface, and a mid-air virtual keyboard with no haptic feedback) and the possible interactions between these factors on typing using the two index fingers in VR. Results show that keypress visual feedback enhanced typing speed and reduced workload, while hover feedback lowered error rates but negatively impacted typing speed. Additionally, using a physical keyboard to provide passive haptic feedback increased the error rate. This increase in the error rate could be attributed to inaccuracies in finger and keyboard tracking, which may have caused a misalignment between the physical and virtual environments. Regarding eye gaze behavior, participants spent more time looking at the keyboard with the keypress visual feedback and when no haptic feedback was provided. Finally, participants rated the physical keyboard as the least usable option.

**Index Terms**—Input techniques, Virtual keyboard, Text entry, Immersive environments

## 1 INTRODUCTION

Typing text on a keyboard is one of the most frequent tasks for content creation activities in the workplace, including document editing, coding, email, web browsing, and social interactions. Different immersive simulation-based applications where learners are involved in typing tasks in usual office environments were developed. These include, for instance, programming or cybersecurity training applications [2, 3, 7].

* Amine Chellali is with the IBISC Lab, Université Evry Paris-Saclay. E-mail: amine.chellali@univ-evry.fr.

* Lucas Herfort is with the IBISC Lab, Université Evry Paris-Saclay. E-mail: lucas.herfort-botel@univ-evry.fr.

* Guillaume Loup is with the IBISC Lab, Université Evry Paris-Saclay. E-mail: guillaume.loup@univ-evry.fr.

* Marie-Hélène Ferrer is with the IBISC Lab, Université Evry Paris-Saclay and The French Armed Forces Biomedical Research Institute. E-mail: marie-helene.ferrer@intradef.gouv.fr.

Manuscript received xx xxx. 201x; accepted xx xxx. 201x. Date of Publication xx xxx. 201x; date of current version xx xxx. 201x. For information on obtaining reprints of this article, please send e-mail to: reprints@ieee.org. Digital Object Identifier: xx.xxxx/TVCG.201x.xxxxxxx

When designing such training environments in VR, offering users comfortable typing conditions is crucial to maintaining task efficiency. This ensures that users focus on the core aspects of the simulation scenario without being distracted by the typing task completion. Ideally, users should seamlessly transfer their real-world typing skills to a virtual environment, moving from a physical keyboard to a virtual one with minimum loss in performance.

Currently, most consumer-grade VR HMDs use indirect methods for text entry, such as controlling a virtual pointer with hand-held controllers or virtual hands for typing on simplified keyboards. While adequate for short tasks like password entry, these methods are unsuitable for complex tasks due to limited performance and the need for training, especially when compared to the familiar standard desktop keyboard [16, 17].

Recent hand and finger tracking advancements in modern HMDs make using standard keyboards in VR environments possible without needing external tracking [35]. However, while this approach is feasible, critical design factors must be investigated as they can impact performance. Furthermore, speed and accuracy alone are insufficient to assess a text entry method [8]. Other metrics, such as usability, learnability, fatigue, workload, and user behavior, must also be accounted for. Further, it is important to consider methods that are not only efficient
but also affordable and easy to set up.

Existing approaches include visualizing physical keyboards within the virtual environment or utilizing virtual keyboards for mid-air interactions. Dube and Arif [8] reviewed VR text entry techniques as of 2019, noting that physical keyboards, which provide natural haptic feedback, typically yield better typing performance [39]. However, they require additional tracking equipment to be properly integrated into the virtual space. Mid-air virtual keyboards, on the other hand, are easier to set up and reposition in the VR environment, but their lack of haptic feedback is often linked to poorer performance [8]. Furthermore, mid-air interactions can lead to increased fatigue, negatively affecting usability.

Various design solutions have been proposed to address these challenges, such as providing visual feedback to compensate for the absence of haptic cues [25, 37] or introducing specialized haptic devices [9, 19]. This study contributes to this ongoing discussion by examining the impact of different visual and/or haptic feedback forms on user experience, gaze behavior, and typing performance in an immersive virtual environment.

## 1.1 Contributions

Our study focuses on implementing virtual keyboards that capture user input via interactions with virtual keys using index fingers, designed for standard HMDs, without relying on additional external hardware. We investigate two types of visual feedback for these keyboards (Fig. 1): one that highlights the key upon hovering with the virtual fingers and another that includes both a highlight and animation upon keypress. In terms of haptic feedback, we compare three conditions (Fig. 1): (i) no haptic feedback with mid-air typing, (ii) passive haptic feedback simulated by typing on a flat physical surface, and (iii) passive haptic feedback typing with a physical keyboard. Our findings show that visual feedback during keypress improved typing speed and reduced workload, while hover feedback led to fewer errors but slowed down typing. Furthermore, using the physical keyboard for passive haptic feedback resulted in higher error rates. Eye gaze behavior was also affected, with participants spending more time looking at the keyboard under the keypress feedback and no haptic feedback conditions. Lastly, the physical keyboard was rated the least usable by participants. We conclude with design implications for text-entry applications in VR.

## 2 RELATED WORK

Our work touches on the areas of virtual keyboards in VR, focusing on haptic and visual feedback cues. Note that our review only discusses the common layouts of desktop keyboards (e.g., Qwerty or similar variants in other languages).

### 2.1 Haptic feedback for VR keyboards

The sense of touch plays a crucial role when typing on a keyboard. With physical keyboards, users feel an opposite force when pressing keys and can rely on them as spatial cues [9]. This feedback is particularly valuable for experienced typists, who use muscle memory to locate keys without looking frequently at the keyboard [12, 33]. The absence of such feedback in virtual keyboards adversely affects typing performance. Various approaches have been developed to provide haptic feedback during typing in VR, which can be categorized into passive and active haptics.

In passive haptics, users are not required to wear specialized devices. Instead, real-world props are used to replicate tactile feedback on virtual objects [28]. Users encounter these physical objects when they interact with the corresponding virtual objects. This approach is applied for text entry in VR by using physical keyboards [5, 16, 21, 26, 32, 35, 39]. In such cases, a 3D virtual model, closely mirroring the physical keyboard's geometry, is meticulously aligned with its real-world counterpart.

This approach presents two key challenges [35]. First, the physical and virtual keyboards must have similar shapes for the experience to be effective. Second, tracking the physical keyboard's position becomes essential, especially in scenarios where both the user and keyboard need to move during the simulation. Addressing this requires external tracking devices, which can be cumbersome and necessitate changes in

the user environment. The effectiveness of this solution also depends heavily on the accuracy and reliability of the tracking sensors [8]. These requirements make physical keyboards impractical for certain scenarios and for low-income users. Nevertheless, they remain the most effective option for text entry in VR, offering typing performance close to real-world levels and providing realistic haptic feedback [9].

To simplify the setup, some researchers have suggested replacing the physical keyboard with a hard, flat surface, such as a table [10]. While this solution provides less realistic haptic interactions with no feedback on key presses, it is appealing due to the reduced need for tracking equipment and easier integration into the user's environment. Dudley et al. [10] have shown that this approach can lead to better performance than typing on a mid-air virtual keyboard.

Researchers have also explored other alternatives, such as active and synthetic haptic devices. These include the use of wearable actuators [19] or haptic gloves [22, 40], which provide haptic and vibrotactile feedback. In another work, Dube and Arif [9] introduced an ultrasound system to simulate haptic feedback during key presses and touches. While these methods offer greater portability, they often involve cumbersome hardware, and user studies suggest that text entry rates remain lower than physical keyboards.

### 2.2 Visual feedback for VR keyboards

Along with haptic feedback, visual feedback plays a crucial role in typing. While experienced typists primarily rely on touch, less skilled users often need to see both the keys and their fingers to type effectively [12, 33]. Visualizing the keyboard helps users familiarize themselves with the key layout while being able to see their hands assists in guiding and maintaining accurate finger placement [16]. Without visual feedback, user experience and performance significantly decline during typing [39]. As discussed earlier, visualizing physical keyboards in virtual environments emphasizes the importance of precise tracking. In the absence of a physical keyboard, mid-air virtual keyboards are used [11]. Their advantage lies in their flexibility, allowing them to appear anywhere in the VR environment, eliminating the need to move to a specific location to use them. However, this flexibility comes at the cost of reduced performance, primarily due to the lack of tactile feedback [9]. Some researchers have successfully implemented auto-correct decoders to improve typing performance [11, 39]. Finally, holding hands in mid-air can quickly become tiring, making this approach unsuitable for prolonged typing sessions.

In addition to visualizing the keyboard, several studies have explored the effects of visualizing users' hands and fingers on typing performance in virtual environments [16, 26, 39]. These studies examined not only the presence or absence of hands' representation but also different forms, including realistic hands, abstract representations, semi-transparent hands, and minimal fingertip visualization. The findings suggest that the absence of hand visualization leads to poorer typing performance and an increased workload [16, 26, 39]. Furthermore, displaying only the fingertips [16] resulted in better performance than full-hands visualization, likely due to the increased occlusion of the virtual keyboard combined with the lack of tracking accuracy in the case of the full-hands. Conversely, Knierim et al. show that semi-transparent hands and fingertips visualization did not significantly affect typing performance. In addition, fingertip visualization decreased the sense of presence and increased the mental workload compared to full-hand visualization for inexperienced typists [26].

Another approach involves using mixed reality to display the physical keyboard and the user's hands via blended videos or point clouds captured by depth cameras [16, 23, 27, 29]. This method has shown promising results, with users achieving near real-world typing performance. However, these techniques introduce added complexity in both hardware and software. They may also disturb the immersion by displaying real-world images inside the virtual environment leading to an occlusion issue [8, 16]. This problem is further amplified by the limited field of view in current HMDs [15].

Finally, in addition to visualizing real-world elements like the keyboard and hands, some research has introduced additional visual cues to augment the virtual keyboard, compensating for the lack of haptic
and visual feedback. For instance, the hover function highlights keys as a finger approaches [9, 17, 19, 24, 25]. In addition, visual feedback on keypress has also been used, usually in association with a key-down animation [9, 16, 19, 39]. Studies on virtual keyboards often highlight the importance of simple, real-time visual feedback over detailed graphics that do not contribute to performance gain [16]. While these cues appear beneficial, their impact on typing performance in virtual environments has yet to be thoroughly evaluated.

## 2.3 Other design considerations

Besides visual and haptic feedback, other design factors in virtual keyboards should be considered. One key aspect is the number of fingers used while typing. Skilled typists typically employ all ten fingers during typing tasks [33]. In contrast, less experienced individuals often rely on fewer fingers, primarily using the two index fingers in a hunt-and-peck style [33]. Interestingly, the study of Dudley et al. [10] found that using all ten fingers led to lower performance than typing with just the two index fingers on a virtual keyboard with mid-air interactions. This performance discrepancy is likely due to finger-tracking limitations in the used HMDs [10], especially when self-occlusion occurs as the hand orientation causes parts of the hand to block the fingers from the camera's view [18]. Another issue with the 10 finger typing was the occurrence of spurious touches due to unintentional finger coactivations [13]. This demonstrates the tracking challenges posed by VR setups with no external tracking systems, which can lead to diminished typing accuracy and speed.

The hunt-and-peck style, where users actively search for keys using their index fingers and often focus on the keyboard rather than the text, introduces another impacting factor—gaze behavior—to typing performance [15]. As discussed earlier, better performance is often associated with reduced visual attention to the keyboard [12]. Novice typists spend more time looking at the keyboard than more experienced typists [12]. Furthermore, forcing users to look at their hands frequently while typing can degrade their performance [38]. Consequently, any design elements that increase the need to look at the keyboard or hands more than users typically would in the real world could negatively impact their typing experience in virtual environments.

To summarize, both haptic and visual feedback play critical roles in enhancing typing performance on virtual keyboards. To achieve a balance between ease of implementation and satisfactory performance, flat physical surfaces emerge as a practical compromise for haptic feedback. They offer minimal passive haptic feedback but outperform mid-air virtual keyboards regarding typing efficiency. Additionally, using physical keyboards with minimal tracking equipment can provide a more realistic haptic experience. However, the impact of tracking accuracy, especially for hand and finger movements, still requires further exploration.

For visual feedback, displaying the virtual keyboard and users' hands is essential for a positive typing experience. However, showing a full-hand representation may hinder performance, primarily due to the limited field of view in current HMDs, which causes the virtual keyboard to become obstructed during typing. Partial hand representations, such as fingertips or semi-transparent hands, can mitigate this issue. Visual aids like highlighting keys during hovering or pressing could further improve the design, though their specific effects on the typing experience and visual attention must be thoroughly evaluated.

Finally, design choices can significantly affect how many fingers users engage while typing, as well as their gaze behavior, both of which have a direct impact on the overall typing experience. This makes these factors critical design considerations.

# 3 USER STUDY

## 3.1 Study objectives

Our research explores various factors influencing performance, gaze behavior, and user experience when typing on a virtual keyboard in immersive environments. The goal is to provide design recommendations for virtual keyboard interactions that can be implemented on standard consumer-grade HMDs, including their native tracking systems, and

without requiring additional hardware. Specifically, we focus on enhancing the virtual keyboard experience by incorporating both visual and haptic cues to compensate for the lack of tactile feedback typically encountered in virtual environments.

We examine two types of visual feedback: key hover highlighting and keypress feedback. Regarding haptics, we compare two types of passive feedback: using a physical keyboard as a prop and interacting with a flat surface. Both conditions are compared to mid-air interaction on a virtual keyboard with no haptic feedback. Unlike prior studies that used physical keyboards for direct text entry, our approach employs the physical keyboard solely as a passive haptic feedback device. Text input is performed through interactions between the virtual fingers and the keyboard. This setup ensures a fair comparison between conditions. It isolates the impact of haptic feedback cues that the user can feel when touching the physical keys while interacting with virtual ones.

This study is exploratory in nature, and we do not assert specific hypotheses. However, we anticipate that combining the physical keyboard with visual cues will enhance typing performance, lower mental workload, and increase usability. Regarding gaze behavior, the physical keyboard is expected to reduce the need to look at the virtual keyboard by providing spatial landmarks. In contrast, the visual feedback cues (hover and keypress highlighting) are likely to draw more attention to the virtual keyboard, increasing the focus on it during typing.

## 3.2 Participants

A total of 24 participants took part in this study, including 7 females and 17 males, recruited among students, university staff, and external participants, with backgrounds in various fields of study. The participants' mean age was $32.75 \pm 12.23$ years (min = 19, max = 59). Three of them were left-handed, 20 were right-handed, and 1 was ambidextrous. All participants had normal or corrected-to-normal vision, with 14 wearing corrective glasses during the experiment. All the participants were native French speakers familiar with Azerty desktop keyboards (French keyboard layout). Note that we intentionally did not control for typing proficiency during recruitment. No specific expertise criteria or prior experience with VR applications were required. Twenty-two of the participants had previous experience with VR headsets, including eight regular users (using them at least once a week). The volunteers have not participated in prior VR typing experiments. The Research Ethics Committee (CER) of Université Paris-Saclay validated the experimental protocol. The study complied with the requisite ethical standards; all participants provided informed written consent before participating.

## 3.3 Experimental design and conditions

We opted for a mixed design in this study to strike a balance between minimizing the number of participants (thereby enhancing statistical power) and reducing participant fatigue caused by repeated trials with the VR HMD. Therefore, the study followed a three-way mixed experimental design involving two within-subject factors: the hover visual feedback with two conditions: activated (Hover on and deactivated (Hover off), and the keypress visual feedback with two conditions: activated (Keypress on) and deactivated (Keypress off), and one between-subjects factor: haptic feedback (HF) with three conditions: physical desktop keyboard (Desktop), hard flat surface (Surface), and no haptic feedback with mid-air interaction (Mid-air).

The hover visual feedback was implemented by changing the color of the top surface of the key to dark blue when the index finger was positioned between 20 mm and 0 mm above the key (Fig. 2). The hover distance was initially set to 25 mm, as suggested by [9]. However, during pilot testing, users felt that the hover feedback was triggered prematurely and preferred a 20 mm distance. The keypress visual feedback consisted of changing the color of the top surface of the key to light blue when the user presses it down (the feedback was generated upon the collision between the virtual finger and the key; Fig. 2). This was also associated with a key-down animation on the virtual keyboard to mimic an actual key press. During the animation, the key moved down 1 mm, similar to key displacement in the physical keyboard. The associated visual feedback was not provided under the "Keypress off" conditions.
<figure><img src="image_3.png" /><figcaption>Fig. 2: Visual feedback provided during the experiment: (top) Hover visual feedback ("Hover on" condition), (bottom) keypress visual feedback ("Keypress on" condition).</figcaption></figure>

In the physical desktop keyboard condition, the virtual keyboard was superposed with the actual keyboard (Fig. 1). In the hard flat surface condition, the virtual keyboard was superposed with a 3D printed model of the same keyboard (with the keys being replaced with a flat plane; Fig. 3). Finally, in the Mid-air condition, the users only used the virtual keyboard and had to type the text mid-air with no physical support below the hands (Fig. 1).

Following our experimental design, the participants were randomly divided into three groups, each assigned to one of the between-subjects factor conditions. Then, within each group, the presentation order of the two within-subjects factor conditions to participants was counter-balanced using a Latin square to mitigate potential learning effects. Participants typed 8 phrases for each within-subjects factor condition. The two first sentences served for training, while the six others served to measure performance. This resulted in a total of 576 recorded analyzed phrases (6 phrases x 2 hover visual feedback conditions x 2 keypress visual feedback conditions x 8 participants x 3 haptic feedback conditions).

## 3.4 Apparatus and virtual environment

### 3.4.1 Real-World Apparatus

The real-world setup served to run a baseline test measuring the participants' typing performance on an actual desktop keyboard. The setup consisted of a Windows PC (with an Intel i7, 16GB RAM, and an Nvidia GTX1080 GPU), an Apple Magic Wireless AZERTY keyboard with physical dimensions of (width x height): 279 x 115 mm, and a 29" monitor (Fig. 3). The computer ran a full-screen typing desktop application developed in the Unity game engine (Version 2021.3.26f1) on a 1920x1080 resolution (Fig. 3).

### 3.4.2 VR Apparatus

For the VR setup, we designed a virtual simplified representation of a room, including the real-world study apparatus comprising the keyboard, the monitor, and the table (Fig. 4). The application was developed using the same Unity version and run on the same PC. A Meta Quest Pro HMD (resolution of 1800x1920 per eye) with hand enabled was used (Fig. 3). The integrated eye tracking system was used to collect eye gaze behavior data. The HMD was connected to the computer with a USB cable using the Meta Quest Link.

In all conditions, the participants saw a virtual representation of their hand displayed in the color of their choice (Fig. 1) to limit the uncanny valley effect [4, 9]. The virtual hand was also displayed in semi-transparency so that the participants could see the keyboard better [16]. The virtual hand model display and tracking were based

on the Meta Quest Pro hand tracking development kit (V2.2). To mitigate the expected finger drift caused by self-occlusion during hand tracking [18], we chose to limit typing to using index fingers only. While this approach was anticipated to slow down text entry compared to real-world typing [33], particularly for skilled typists, we based our decision on findings from Dudley et al. [10], which showed that typing with just the index fingers on a virtual keyboard yields better performance than using all ten fingers.

The real and virtual keyboards were aligned in the physical keyboard and flat surface conditions. This required a calibration process using the Meta Quest Pro controller. The process began by placing the physical keyboard in a fixed position relative to the controller. A custom board with a 3D-printed controller support was designed for this purpose (Fig. 3). The user wore the HMD to start the calibration, and the experimenter pressed a controller button. This captured the position and orientation of the controller in relation to the HMD and adjusted the location of the virtual keyboard to align with the physical one. Note that participants had to wait for the controller to enter sleep mode before hand interactions could begin. After calibration, users confirmed the alignment of the virtual and physical keyboard surfaces. If misalignment occurred, the procedure was repeated until a correct alignment was achieved.

In the virtual keyboard condition, the calibration process was performed so that the virtual keyboard was in the same position. However, the board was removed after calibration to allow mid-air interaction with the keyboard (Fig. 1).

Note that the system was re-calibrated after each condition to ensure alignment between the virtual and physical keyboards.

## 3.5 Stimulus and experimental task

### 3.5.1 Stimulus

Stimulus sentences were drawn from the Multilingual Phrase set based on movie captions [14]. Indeed, Franco-Salvador and Leiva [14] argue that native phrase sets should be used for text entry tasks. Therefore, we used their French phrase set. The phrases contained only the letters a–z. During the experiment, the participants were asked to type two sentences for training (not repeated for the typing test) and six random sentences for the typing test to measure performance and user experience with each condition.

### 3.5.2 Text entry task

For each condition, participants performed a copy text task with native phrases, as described above. They were shown stimulus phrases randomly drawn from the dataset, one at a time. They were asked to type them as quickly and as accurately as possible. Error correction using the backspace key was not allowed. This was done in line with previous studies [16,39]. Once a sentence was typed, the participants had to put their virtual hand on a validation button (Fig. 4) to move to the following sentence (In the real-world setup, they used the mouse to click on a validation widget displayed on the screen, Fig. 3).

## 3.6 Experimental procedure

After arriving at the experimental room, the participants were introduced to the study's objective and setup. Next, they were asked to read and sign a consent form to participate in the study. Then, they were given an instruction sheet detailing how the setup functions, the tasks to be performed, and what was expected of them. The subsequent step was to complete a demographic questionnaire.

The actual experiment started with performing the real-world setup's baseline test. After that, the participants were asked to put on the HMD to begin with the first VR condition. The conditions were separated by a 5-minute break, during which participants filled out the subjective questionnaires. On average, participants completed the study in 40 minutes.

## 3.7 Measurements and data analyses

Performance was measured based on text entry rate, error rate, and time to the first key press. Text entry rate is calculated in Words Per Minute (WPM) with a word defined as five consecutive characters, including
<figure><img src="image_4.png" /><figcaption>Fig. 3: Apparatus used during the experiment. (A) Real-word apparatus, (B) VR apparatus, (C) Physical keyboard (Desktop condition), (D) 3D printed flat surface (Surface condition) with the same shape and size as the physical keyboard.</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>Fig. 4: The virtual environment displayed during the experiment.</figcaption></figure>

spaces [36]. We measured time from the first key press to the click on the validation button. For several applications, the time to react to a specific event using the keyboard input is a critical measure of typing performance. Therefore, we investigated the time to the first key press. This metric was first introduced by McGill et al. [29]. The error rate was measured as the character error rate (CER). CER, as defined by Gubert et al. [16], is the minimum number of character-level insertion, deletion, and substitution operations required to transform the response text into the stimulus text, divided by the number of characters in the stimulus text.

In addition, we measured the eye gaze behavior. This was based on the visual attention defined by Feit et al. [12] as the ratio between the time spent looking at the keyboard and the time a sentence was displayed (from the display of the stimuli sentence to the click on the validation button). Between 0 and 100%, where 0 means no time spent looking at the keyboard.

Subjective measurements included the NASA TLX to measure mental workload [20], and the System Usability Scale [6].

The statistical data analysis was performed using SPSS software, with a confidence level of 95%. Shapiro-Wilk tests were used to check data normal distributions. Levene's tests were used to assess the homogeneity of variances. Based on these preliminary checks, 3-way mixed-design ANOVAs were used. Finally, pairwise comparisons with Bonferroni adjustments were used when necessary.

# 4 RESULTS

## 4.1 Objective measurements

The results of the three-way ANOVAs for the objective measures are listed in Table 1.

### 4.1.1 Pre-test results

First, we checked whether the real-world typing performance was comparable between the three experimental groups (according to the between-subject factor). The results of the one-way ANOVA indicate no significant difference between the groups ($F_{(2,21)} = 2.63, p > .05$). The average entry rate was $45.37 \pm 7.44, 57.32 \pm 11.90$, and $50.35 \pm 11.43$, for the Physical Keyboard, the Flat Surface and the Mid-air conditions respectively.

We checked also whether the real-world error rate was comparable between the three experimental groups. The results of the one-way ANOVA indicate no significant difference between the groups ($F_{(2,21)} = 1.10, p > .05$). The average error rate was $.68 \pm .64, .65 \pm .49$, and $.46 \pm .33$, for the Physical Keyboard, the Flat Surface and the Mid-air conditions respectively.

### 4.1.2 Entry rate

A three-way mixed-design ANOVA revealed a significant interaction effect between the keypress visual feedback and hover visual feedback on WPM ($F_{(1,21)} = 7.67, p = .01$, partial $\eta^2 = .27$). A significant main effect of hover visual feedback on WPM was also observed ($F_{(1,21)} = 16.99, p < .0001$, partial $\eta^2 = .44$). However, the main effect of haptic feedback on WPM was not significant ($F_{(2,21)} = 1.19, p = .32$, partial $\eta^2 = .10$). No other significant main or interaction effects were observed.

The participants performed better when the keypress visual feedback was activated and the hover visual feedback was deactivated. On the other hand, the performance decreased significantly when the hover feedback was activated simultaneously with the keypress visual feedback (Fig. 5).

### 4.1.3 Time to first key press

A three-way mixed-design ANOVA revealed a significant main effect of hover visual feedback on the time to first key press ($F_{(1,21)} = 13.38, p = .001$, partial $\eta^2 = .39$). No other significant main or interaction effects were observed.

The participants pressed the first key faster when the hover visual feedback was deactivated (Fig. 6).

### 4.1.4 Error rate

A three-way mixed-design ANOVA revealed a significant main effect of hover visual feedback on the CER ($F_{(1,21)} = 10.68, p = .004$, partial $\eta^2 = .33$), and a significant main effect of haptic feedback on the CER ($F_{(2,21)} = 4.57, p = .02$, partial $\eta^2 = .30$). No other significant main or interaction effects were observed.

The participants performed better when the hover visual feedback was activated (Fig. 7). In addition, the pairwise comparisons indicate that the participants performed better in the Mid-air condition than in the physical keyboard condition (adjusted-$p = .007$).
<table><caption>Table 1: ANOVA results for text entry rate (WPM), time to first key press (1KP), character error rate (CER%), and visual attention (ViA). \* = p < .05; \*\* = p < .01; \*\*\* = p < .001.</caption><thead><tr><th rowspan="2">Effect</th><th rowspan="2">df</th><th colspan="3">WPM</th><th colspan="3">1KP</th><th colspan="3">CER%</th><th colspan="3">ViA</th></tr><tr><th>F</th><th>η<sup>2</sup></th><th>p-value</th><th>F</th><th>η<sup>2</sup></th><th>p-value</th><th>F</th><th>η<sup>2</sup></th><th>p-value</th><th>F</th><th>η<sup>2</sup></th><th>p-value</th></tr></thead><tbody><tr><td>HF</td><td>2, 21</td><td>1.19</td><td>.10</td><td>.32</td><td>2.99</td><td>.22</td><td>.072</td><td>4.57*</td><td>.30</td><td>.02</td><td>4.07*</td><td>.28</td><td>.03</td></tr><tr><td>Hover (H)</td><td>1, 21</td><td>.61</td><td>.03</td><td>.44</td><td>13.38**</td><td>.39</td><td>.001</td><td>10.68**</td><td>.33</td><td>.004</td><td>1.19</td><td>.05</td><td>.28</td></tr><tr><td>Keypress (K)</td><td>1, 21</td><td>16.99***</td><td>.44</td><td>&lt;.0001</td><td>.08</td><td>.004</td><td>.76</td><td>1.72</td><td>.07</td><td>.20</td><td>11.02**</td><td>.34</td><td>.003</td></tr><tr><td>HF × H</td><td>2, 21</td><td>.24</td><td>.02</td><td>.78</td><td>0.42</td><td>.03</td><td>.66</td><td>1.20</td><td>.10</td><td>.31</td><td>1.93</td><td>.15</td><td>.17</td></tr><tr><td>HF × K</td><td>2, 21</td><td>0.91</td><td>.08</td><td>.41</td><td>.81</td><td>.07</td><td>.45</td><td>.31</td><td>.03</td><td>.73</td><td>0.51</td><td>.04</td><td>.60</td></tr><tr><td>H × K</td><td>2, 21</td><td>7.67*</td><td>.26</td><td>.011</td><td>.74</td><td>.03</td><td>.39</td><td>.25</td><td>.01</td><td>.61</td><td>0.12</td><td>.006</td><td>.72</td></tr><tr><td>HF × H × K</td><td>2, 21</td><td>.97</td><td>.08</td><td>.39</td><td>.88</td><td>.07</td><td>.42</td><td>.24</td><td>.02</td><td>.78</td><td>0.41</td><td>.03</td><td>.67</td></tr></tbody></table>

<figure><img src="image_6.png" /><figcaption>Fig. 5: Text entry rate measured in words per minute (Higher is better, error bars represent the standard error). A significant main effect of Hover visual feedback and an interaction between Hover and Keypress visual feedback are observed (\* = p < .05; \*\*\* = p < .001).</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>Fig. 6: Time to first key press (Lower is better, error bars represent the standard error). A significant main effect of the hover feedback was observed (\*\* = p < .01).</figcaption></figure>

### 4.1.5 Eye gaze behavior

A three-way mixed-design ANOVA revealed a significant main effect of keypress visual feedback ($F_{(1,21)} = 11.02$, p = .003, partial $\eta^2$ = .34), and a significant main effect of the haptic feedback condition on visual attention ($F_{(2,21)} = 4.07$, p = .03, partial $\eta^2$ = .28). No other significant main or interaction effects were observed.

When the keypress visual feedback was activated, the participants spent significantly more time looking at the keyboard than when it was deactivated (Fig. 8). In addition, the pairwise comparisons indicate that the participants spent significantly more time looking at the keyboard in the Mid-air condition than in the physical keyboard condition (adjusted-p = .04).

<table><caption>Table 2: ANOVA results for the mental workload (NASA-TLX) and the usability (SUS). \* = p < .05; \*\* = p < .01.</caption><thead><tr><th rowspan="2">Effect</th><th rowspan="2">df</th><th colspan="3">NASA-TLX</th><th colspan="3">SUS</th></tr><tr><th>F</th><th>η<sup>2</sup></th><th>p</th><th>F</th><th>η<sup>2</sup></th><th>p</th></tr></thead><tbody><tr><td>HF</td><td>2, 21</td><td>.63</td><td>.05</td><td>.53</td><td>3.87*</td><td>.27</td><td>.03</td></tr><tr><td>H</td><td>1, 21</td><td>.33</td><td>.01</td><td>.56</td><td>.04</td><td>.002</td><td>.83</td></tr><tr><td>K</td><td>1, 21</td><td>13.1**</td><td>.38</td><td>.002</td><td>1.46</td><td>.06</td><td>.24</td></tr><tr><td>HF×H</td><td>2, 21</td><td>.46</td><td>.04</td><td>.63</td><td>0.16</td><td>.01</td><td>.85</td></tr><tr><td>HF×K</td><td>2, 21</td><td>2.09</td><td>.16</td><td>.14</td><td>.28</td><td>.02</td><td>.75</td></tr><tr><td>H×K</td><td>2, 21</td><td>0.07</td><td>.003</td><td>.79</td><td>.13</td><td>.006</td><td>.71</td></tr><tr><td>HF×H×K</td><td>2, 21</td><td>.49</td><td>.04</td><td>.61</td><td>.96</td><td>.08</td><td>.39</td></tr></tbody></table>

## 4.2 Subjective measures

The results of the three-way ANOVAs for the subjective measures are listed in Table 2.

### 4.2.1 NASA-TLX

A three-way mixed-design ANOVA revealed a significant main effect of keypress visual feedback on the NASA-TLX raw scores ($F_{(1,21)} = 13.10$, p = .002, partial $\eta^2$ = .38). No other significant main effects or interactions were observed.

The NASA-TLX scores were significantly lower when the keypress visual feedback was activated (Fig. 9).

### 4.2.2 Usability

A three-way mixed-design ANOVA revealed a significant main effect of the haptic feedback condition on the SUS scores ($F_{(2,21)} = 3.87$, p = .03, partial $\eta^2$ = .27). No other significant main effects or interactions were observed.

The pairwise comparisons indicate that the usability scores (Fig. 10) were significantly higher in the Mid-air condition than in the physical keyboard condition (adjusted-p = .04). No other significant difference was found.

## 5 DISCUSSION

Our results offer a deeper understanding of how visual and haptic feedback impact typing performance in immersive virtual environments. Specifically, our findings illustrate the distinct effects of different feedback types on speed, accuracy, workload, and usability, as well as the interaction between these factors, highlighting critical trade-offs that VR interface designers must address. In addition, we provide new data on eye gaze behavior. While this was previously studied in the
<figure><img src="image_8.png" /><figcaption>Fig. 7: Character error rate (Lower is better, error bars represent the standard error). Significant main effects of the hover visual feedback and the haptic feedback are observed (** = p < .01).</figcaption></figure>

<figure><img src="image_9.png" /><figcaption>Fig. 8: The ratio of time the participants spent looking at the virtual keyboard during typing (error bars represent the standard error). Note that the vertical axes do not start at zero. Significant main effects of the keypress visual feedback and the haptic feedback are observed (* = p < .05; ** = p < .01).</figcaption></figure>

real world, to the best of our knowledge, no previous studies reported eye gaze behavior data while typing text in an immersive environment. This measure provides additional insights into how users interact with virtual keyboards.

## 5.1 Effect of Visual Feedback

Our results show that the keypress visual feedback significantly increased the typing speed and reduced cognitive workload. While the VR typing performance remains below that observed in real-world typing (using ten fingers), this finding suggests that the real-time visual confirmation of a keypress can improve two-finger typing performance in VR by compensating for the absence of tactile cues. This is consistent with previous research [8] emphasizing the importance of immediate feedback for enhancing typing performance in the absence of haptic cues. The NASA-TLX scores in our study further confirm that keypress visual feedback helps reduce the mental and physical effort required during typing tasks, leading to smoother and more efficient interactions.

Conversely, the hover visual feedback exhibited more nuanced effects. While it contributed to reducing error rates, it also resulted in slower typing speeds, particularly when associated with the keypress visual feedback. One possible explanation is that participants became overly reliant on the hover cue to anticipate keypresses, which required

<figure><img src="image_10.png" /><figcaption>Fig. 9: The NASA-TLX scores (Lower is better; error bars represent the standard error) A significant main effect of the keypress visual feedback is observed (** = p < .01).</figcaption></figure>

<figure><img src="image_11.png" /><figcaption>Fig. 10: The scores of the System Usability Scale (Higher is better, error bars represent the standard error). A significant main effect of the haptic feedback is observed (* = p < .05).</figcaption></figure>

them to focus more on placing their virtual fingers above the keyboard. This is further confirmed by the time before the first keypress, which increased when the Hover function was activated, suggesting an increase in the processing time before deciding to press a key.

In practice, this means that hover feedback, while helpful for improving accuracy, may detract from the user's overall ability to multitask or focus on other elements of the VR environment. From a design perspective, the hover feedback introduces a level of caution, encouraging users to focus on accuracy at the expense of speed. This suggests that hover feedback is most useful in tasks where accuracy is prioritized, such as entering passwords or other high-precision tasks, but less ideal for tasks that require rapid typing.

In addition, the significant interaction effect between hover and keypress visual feedback raises interesting questions about multiple feedback integration. Our results indicate that activating hover and keypress feedback simultaneously decreased performance. This suggests that combining these two feedback mechanisms can overwhelm users, requiring them to process too much information simultaneously. Thus, care must be taken when combining different types of feedback, as overloading users with multiple signals from the same modality can detract from their performance rather than enhance it. Alternatives for future designs could involve adaptive feedback systems that adjust the type and amount of feedback based on the user's typing speed, task difficulty, or personal preference.

Finally, as expected, the results of visual attention measurements indicate an increase in visual attention toward the keyboard when the
keypress feedback was activated. This confirms that visual cues attract the user's gaze. Surprisingly, this was not the case for the hover visual feedback. Future studies should be conducted to isolate this factor and better understand its impact on the visual behavior during typing in VR.

## 5.2 Effect of Haptic Feedback

Contrary to expectations, passive haptic feedback provided by the physical keyboard did not result in better performance, as it led to higher error rates. A likely cause for the increased error rate is the misalignment between virtual and physical keyboards due to tracking errors despite our calibration process. Our calibration was based on the tracking accuracy of the Meta Quest Pro controller. Unfortunately, we did not find studies reporting accuracy data for this device. However, we believe that even slight alignment discrepancies, where virtual keys do not perfectly match the physical keyboard, can cause users to mistype, particularly in a visually demanding VR context.

This misalignment introduces spatial uncertainty, possibly explaining why the physical keyboard condition led to higher error rates. This may also explain the scores of the usability questionnaire. Contrary to expectations, the physical keyboard obtained the lowest usability scores. One possible explanation of these results can be found in the work of McMahan et al. [30] describing the phenomenon of the uncanny valley of VR interactions. The authors hypothesize that increasing interaction fidelity from low interaction fidelity techniques will initially result in worse user performances. As interaction fidelity continues to increase, and the overall degree of fidelity becomes relatively high, user performance will rebound and be comparable, if not better, than those afforded by the low-fidelity techniques [30]. In our case, text input using the physical keyboard was expected to increase the interaction fidelity by providing more realistic haptic feedback on the keypress. However, the tracking issues with misalignment decreased the interaction fidelity, leading to a lower performance. Additionally, users may have had higher expectations when interacting with the physical keyboard, as it closely resembles their everyday typing experience. The discrepancy between their anticipated performance and the actual lower accuracy might have led to disappointment, resulting in low usability ratings. This reaction is similar to the uncanny valley phenomenon observed with robots [31].

On the other hand, mid-air typing with no haptic feedback yielded better accuracy than the physical keyboard. This suggests that users may rely more on visual feedback to guide their interactions in the absence of conflicting spatial cues, resulting in fewer errors. This finding has important implications for low-cost VR systems or scenarios where physical keyboards are impractical or unavailable. The absence of haptic feedback may not always hinder performance and, in fact, could be beneficial in reducing error rates when combined with appropriate visual feedback mechanisms.

Finally, the flat surface condition provided a simpler form of passive haptic feedback and comparable speed and error rate results with those obtained in the mid-air interaction condition. The flat surface offers a compromise between the no-haptic and physical keyboard conditions, providing lower fidelity haptic feedback but without the complexity of aligning a full keyboard. While this condition did not outperform the others, it may represent a practical solution for portable or shared VR systems, where space constraints or hardware limitations make physical keyboards impractical. Flat surfaces are easier to integrate into different environments and provide users with some spatial cues without requiring precise calibration. It can also reduce fatigue that may disturb users during long typing sessions with mid-air interaction keyboards.

Note that the Mid-air and flat surface conditions yield comparable usability scores, reflecting better user acceptability of these lower interaction fidelity techniques, in which they also performed better. In general, the usability results highlight the need for flexible feedback mechanisms that can be adjusted to suit different user preferences and task requirements. Given the diversity of user experiences with VR and typing in general, one-size-fits-all solutions are unlikely to provide the best user experience. Instead, VR systems could benefit from person-

alized settings that allow users to toggle between different feedback modes or adjust the visual and haptic feedback levels to optimize their performance and comfort.

Finally, the results of gaze behavior are also noteworthy. As expected, participants spent significantly more time looking at the virtual keyboard in the Mid-air condition than in the physical keyboard condition, likely due to the absence of tactile cues to guide their hand placement. This suggests that visual feedback becomes increasingly important when physical feedback is removed, as users must rely entirely on their visual sense to perform the typing task. However, in VR simulation applications, where continuous focus on the task is necessary, this increased reliance on visual feedback may introduce fatigue or distraction over extended periods. From the design of simulation-based training systems perspective, this distraction may hinder the smooth running of the learning scenarios.

## 5.3 Limitations and future work

The duration of experimental sessions was relatively short (40 minutes on average), with 5-minute breaks between conditions. This may have reduced fatigue risk. Fatigue was reported in previous studies with mid-air keyboard interactions as one of the major challenges. With longer sessions, one can expect the appearance of muscle fatigue, which may degrade the performance and the user experience in the Mid-air condition. This issue should be investigated in future studies.

We chose two shades of blue for the visual feedback aids provided during the experiment (hover and keypress). Although no participants reported any confusion between the two used colors, we acknowledge that using more distinct colors for these factors could be beneficial in the future. This could help further enhance the clarity of visual aids.

For the sake of comparability between the haptic feedback conditions, the text entry in the physical was based on touch interactions on the virtual keyboard using the virtual hands. Instead, previous studies [5, 16, 21, 26, 32, 35, 39] used text inputs from the physical keyboards with the real hands even though virtual keyboards were displayed. The input accuracy was inevitably negatively impacted by misalignment between the physical and virtual keyboards and hand-tracking errors. With our design requirements of using only the native tracking systems of commercial HMDs, it will be interesting in the future to assess the typing performance with direct inputs on the physical keyboard using the same calibration strategy.

The overall typing performance in all conditions was much lower than that of the real-world baseline. While this is also lower than typing performance observed in previous studies that support interactions with physical keyboards [5, 16, 21, 26, 32, 35, 39], it is comparable to that observed in studies with mid-air interaction on virtual keyboards [8]. The duration of experimental sessions was short and may have limited the learning effect of using each setup. Another factor that may have decreased the performance, particularly for experienced typists, is using two fingers instead of ten fingers for typing. We expect users to improve their performance with more practice to attain higher typing speed levels. We also note that this performance is relative to text entry using two fingers. In future studies, it will be important to evaluate typing performance using ten fingers. This would better reflect the typing style of expert typists and could influence their use of visual cues such as Hovering feedback.

Previous research has reported varying levels of finger-tracking accuracy when using commercial VR head-mounted displays equipped with multiple built-in cameras. For example, Schneider et al. [34] observed lower spatial accuracy for HTC Vive hand tracking compared to Oculus Quest and Leap Motion, with estimation errors of 37 mm, 16 mm, and 13 mm, respectively. More recently, Abdikarim et al. [1] introduced a framework for measuring tracking accuracy in VR hand-tracking systems, reporting an average fingertip position error of 11 mm for the Oculus Quest 2.

In our study, we used the Oculus Quest Pro HMD with the hand-tracking development kit V2.2. Unfortunately, specific accuracy data on this version is not available. However, it is reasonable to assume that comparable tracking errors persist even with current state-of-the-art hardware, which may have contributed to the reduced typing perfor-
mance observed in our study. Future work should incorporate accuracy measurements to better evaluate typing performance on virtual keyboards.

Visual attention toward the keyboard was much higher in all conditions (around 88% on average) compared to data reported in previous studies on real-world typing (around 20% for expert typists and 41% for novice typists) [12]. Prior research also indicates that expert typists use vision to monitor the hands to inhibit inappropriate keystrokes when typing in unusual conditions and rely more on haptic feedback in normal typing conditions [38]. As all the participants were unfamiliar with typing text in VR, this may explain the increased visual attention toward the keyboard. This can also partially explain the high drop in performance (for both accuracy and typing speed) between the baseline condition in the real world and all the VR conditions in our study. Again, with training, users can become more familiar with the typing conditions in VR and thus look less often toward the keyboard and hands.

# 6 CONCLUSION

This study advances the understanding of how visual and haptic feedback affect typing speed, accuracy, usability, and visual attention in immersive virtual environments. The findings reveal a clear trade-off between speed and accuracy in virtual keyboards' design, particularly regarding visual feedback cues. While keypress feedback improves typing speed and reduces workload, hover feedback is more effective in reducing errors but comes at the cost of slower performance. This suggests that different tasks may benefit from different feedback configurations, depending on whether speed or accuracy is prioritized.

On the other hand, realistic haptic feedback via a physical keyboard increased the error rates likely due to misalignment with the virtual keyboard. In scenarios where physical keyboards are impractical, mid-air typing with no haptic feedback emerges as a viable alternative. The findings also highlight the importance of eye gaze behavior in assessing visual attention, with participants spending more time looking at the keyboard in conditions where haptic feedback was absent or where keypress feedback was active. This suggests that VR systems should carefully consider the balance between visual and haptic feedback to avoid overburdening users' visual attention.

In conclusion, our study highlights the need for flexible, adaptive feedback systems in VR environments. Designers should consider offering users the ability to customize or toggle feedback settings based on the task's specific requirements or the user's preferences. Future research could explore the development of adaptive systems that dynamically adjust feedback modes based on real-time user performance or environmental conditions, thereby improving both usability and performance across a wider range of applications. Additionally, longitudinal studies are needed to assess how users adapt to feedback mechanisms over time, providing further insights into virtual typing systems' long-term usability and learning curves.

# ACKNOWLEDGMENTS

The authors would like to thank all the volunteers who participated in the study. The authors would also like to thank Lucas Beven and Christophe Luquin for their assistance in designing the study apparatus. This work was supported by a grant from the French National Research Agency (ANR-21-CE39-0015 DRIFT-FH).

# REFERENCES

[1] D. Abdlkarim, M. Di Luca, P. Aves, M. Maaroufi, S.-H. Yeo, R. C. Miall, P. Holland, and J. M. Galea. A methodological framework to assess the accuracy of virtual reality hand-tracking systems: A case study with the meta quest 2. *Behavior research methods*, 56(2):1052–1063, 2024. 8

[2] F. Abu Deeb. Enhancing cybersecurity with extended reality: A systematic review. *Journal of Computer Information Systems*, pp. 1–15, 2024. 1

[3] A. M. Alnajim, S. Habib, M. Islam, H. S. AlRawashdeh, and M. Wasim. Exploring cybersecurity education and training techniques: a comprehensive review of traditional, virtual reality, and augmented reality approaches. *Symmetry*, 15(12):2175, 2023. 1

[4] F. Argelaguet, L. Hoyet, M. Trico, and A. Lécuyer. The role of interaction in virtual embodiment: Effects of the virtual hand representation. In *2016 IEEE virtual reality (VR)*, pp. 3–10. IEEE, 2016. 4

[5] S. Bovet, A. Kehoe, K. Crowley, N. Curran, M. Gutierrez, M. Meisser, D. O. Sullivan, and T. Rouvinez. Using traditional keyboards in vr: Steamvr developer kit and pilot game user study. In *2018 IEEE Games, Entertainment, Media Conference (GEM)*, pp. 1–9. IEEE, 2018. 2, 8

[6] J. Brooke. Sus: A quick and dirty usability scale. *Usability Evaluation in Industry/Taylor and Francis*, 1996. 5

[7] J. Dominic, B. Tubre, D. Kunkel, and P. Rodeghero. The human experience of comprehending source code in virtual reality. *Empirical Software Engineering*, 27(7):173, 2022. 1

[8] T. J. Dube and A. S. Arif. Text entry in virtual reality: A comprehensive review of the literature. In *Human-Computer Interaction. Recognition and Interaction Technologies: Thematic Area, HCI 2019, Held as Part of the 21st HCI International Conference, HCII 2019, Orlando, FL, USA, July 26–31, 2019, Proceedings, Part II 21*, pp. 419–437. Springer, 2019. 1, 2, 7, 8

[9] T. J. Dube and A. S. Arif. Ultrasonic keyboard: A mid-air virtual qwerty with ultrasonic feedback for virtual reality. In *Proceedings of the Seventeenth International Conference on Tangible, Embedded, and Embodied Interaction*, pp. 1–8, 2023. 2, 3, 4

[10] J. Dudley, H. Benko, D. Wigdor, and P. O. Kristensson. Performance envelopes of virtual keyboard text input strategies in virtual reality. In *2019 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)*, pp. 289–300. IEEE, 2019. 2, 3, 4

[11] J. J. Dudley, J. Zheng, A. Gupta, H. Benko, M. Longest, R. Wang, and P. O. Kristensson. Evaluating the performance of hand-based probabilistic text input methods on a mid-air virtual qwerty keyboard. *IEEE Transactions on Visualization and Computer Graphics*, 2023. 2

[12] A. M. Feit, D. Weir, and A. Oulasvirta. How we type: Movement strategies and performance in everyday typing. In *Proceedings of the 2016 chi conference on human factors in computing systems*, pp. 4262–4273, 2016. 2, 3, 5, 9

[13] C. R. Foy, J. J. Dudley, A. Gupta, H. Benko, and P. O. Kristensson. Understanding, detecting and mitigating the effects of coactivations in ten-finger mid-air typing in virtual reality. In *Proceedings of the 2021 CHI conference on human factors in computing systems*, pp. 1–11, 2021. 3

[14] M. Franco-Salvador and L. A. Leiva. Multilingual phrase sampling for text entry evaluations. *International Journal of Human-Computer Studies*, 113:15–31, 2018. 4

[15] M. Gonzalez-Franco, D. Abdlkarim, A. Bhatia, S. Macgregor, J. A. Fotso-Puepi, E. J. Gonzalez, H. Seifi, M. Di Luca, and K. Ahuja. Hovering over the key to text input in xr. *arXiv preprint arXiv:2406.09579*, 2024. 2, 3

[16] J. Grubert, L. Witzani, E. Ofek, M. Pahud, M. Kranz, and P. O. Kristensson. Effects of hand representations for typing in virtual reality. In *2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR)*, pp. 151–158. IEEE, 2018. 1, 2, 3, 4, 5, 8

[17] J. Grubert, L. Witzani, E. Ofek, M. Pahud, M. Kranz, and P. O. Kristensson. Text entry in immersive head-mounted display-based virtual reality using standard keyboards. In *2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR)*, pp. 159–166. IEEE, 2018. 1, 3

[18] Y. Gu, C. Yu, Z. Li, Z. Li, X. Wei, and Y. Shi. Qwertying: Text entry on physical surfaces using a ring. *Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies*, 4(4):1–29, 2020. 3, 4

[19] A. Gupta, M. Samad, K. Kin, P. O. Kristensson, and H. Benko. Investigating remote tactile feedback for mid-air text-entry in virtual reality. In *2020 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)*, pp. 350–360. IEEE, 2020. 2, 3

[20] S. Hart. Development of nasa-tlx (task load index): Results of empirical and theoretical research. *Human mental workload/Elsevier*, 1988. 5

[21] A. H. Hoppe, L. Otto, F. van de Camp, R. Stiefelhagen, and G. Unmübig. qvrty: Virtual keyboard with a haptic, real-world representation. In *HCI International 2018–Posters’ Extended Abstracts: 20th International Conference, HCI International 2018, Las Vegas, NV, USA, July 15-20, 2018, Proceedings, Part II 20*, pp. 266–272. Springer, 2018. 2, 8

[22] S. Jadhav, V. Kannanda, B. Kang, M. T. Tolley, and J. P. Schulze. Soft robotic glove for kinesthetic haptic feedback in virtual reality environments. *electronic imaging*, 29:19–24, 2017. 2

[23] H. Jiang, D. Weng, Z. Zhang, Y. Bao, Y. Jia, and M. Nie. Hikeyb: High-efficiency mixed reality system for text entry. In *2018 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)*, pp. 132–137. IEEE, 2018. 2
[24] S. Kato and B. Shizuki. Hover-based japanese input method without restricting arm position for xr. In Proceedings of the 35th Australian Computer-Human Interaction Conference, pp. 564–572, 2023. 3

[25] Y. R. Kim and G. J. Kim. Hovr-type: Smartphone as a typing interface in vr using hovering. In Proceedings of the 22nd ACM Conference on Virtual Reality Software and Technology, pp. 333–334, 2016. 2, 3

[26] P. Knierim, V. Schwind, A. M. Feit, F. Nieuwenhuizen, and N. Henze. Physical keyboards in virtual reality: Analysis of typing performance and effects of avatar hands. In Proceedings of the 2018 CHI conference on human factors in computing systems, pp. 1–9, 2018. 2, 8

[27] J.-W. Lin, P.-H. Han, J.-Y. Lee, Y.-S. Chen, T.-W. Chang, K.-W. Chen, and Y.-P. Hung. Visualizing the keyboard in virtual reality for enhancing immersive experience. In ACM SIGGRAPH 2017 Posters, pp. 1–2. ACM, 2017. 2

[28] R. W. Lindeman, J. L. Sibert, and J. K. Hahn. Hand-held windows: towards effective 2d interaction in immersive virtual environments. In Proceedings IEEE Virtual Reality (Cat. No. 99CB36316), pp. 205–212. IEEE, 1999. 2

[29] M. McGill, D. Boland, R. Murray-Smith, and S. Brewster. A dose of reality: Overcoming usability challenges in vr head-mounted displays. In Proceedings of the 33rd annual ACM conference on human factors in computing systems, pp. 2143–2152, 2015. 2, 5

[30] R. P. McMahan, C. Lai, and S. K. Pal. Interaction fidelity: the uncanny valley of virtual reality interactions. In Virtual, Augmented and Mixed Reality: 8th International Conference, VAMR 2016, Held as Part of HCI International 2016, Toronto, Canada, July 17-22, 2016. Proceedings 8, pp. 59–70. Springer, 2016. 8

[31] M. Mori. Bukimi no tani [the uncanny valley]. Energy, 7:33, 1970. 8

[32] D.-M. Pham and W. Stuerzlinger. Hawkey: Efficient and versatile text entry for virtual reality. In Proceedings of the 25th ACM Symposium on Virtual Reality Software and Technology, pp. 1–11, 2019. 2, 8

[33] S. Pinet, C. Zielinski, F.-X. Alario, and M. Longcamp. Typing expertise in a large student population. Cognitive Research: Principles and Implications, 7(1):77, 2022. 2, 3, 4

[34] D. Schneider, V. Biener, A. Otte, T. Gesslein, P. Gagel, C. Campos, K. Čopič Pucihar, M. Kljun, E. Ofek, M. Pahud, et al. Accuracy evaluation of touch tasks in commodity virtual and augmented reality head-mounted displays. In Proceedings of the 2021 ACM Symposium on Spatial User Interaction, pp. 1–11, 2021. 8

[35] D. Schneider, A. Otte, T. Gesslein, P. Gagel, B. Kuth, M. S. Damlakhi, O. Dietz, E. Ofek, M. Pahud, P. O. Kristensson, et al. Reconviguration: Reconfiguring physical keyboards in virtual reality. IEEE transactions on visualization and computer graphics, 25(11):3190–3201, 2019. 1, 2, 8

[36] R. W. Soukoreff and I. S. MacKenzie. Metrics for text entry research: An evaluation of msd and kspc, and a new unified error metric. In Proceedings of the SIGCHI conference on Human factors in computing systems, pp. 113–120, 2003. 5

[37] M. Speicher, A. M. Feit, P. Ziegler, and A. Krüger. Selection-based text entry in virtual reality. In Proceedings of the 2018 CHI conference on human factors in computing systems, pp. 1–13, 2018. 2

[38] K. M. Tapp and G. D. Logan. Attention to the hands disrupts skilled typewriting: The role of vision in producing the disruption. Attention, Perception, & Psychophysics, 73(8):2379–2383, 2011. 3, 9

[39] J. Walker, B. Li, K. Vertanen, and S. Kuhl. Efficient typing on a visually occluded physical keyboard. In Proceedings of the 2017 CHI conference on human factors in computing systems, pp. 5457–5461, 2017. 2, 3, 4, 8

[40] C.-M. Wu, C.-W. Hsu, T.-K. Lee, and S. Smith. A virtual reality keyboard with realistic haptic feedback in a fully immersive virtual environment. Virtual Reality, 21:19–29, 2017. 2