# Effects of End-to-end Latency on User Experience and Performance in Immersive Virtual Reality Applications

Polona Caserman, Michelle Martinussen, Stefan Göbel

► To cite this version:

Polona Caserman, Michelle Martinussen, Stefan Göbel. Effects of End-to-end Latency on User Experience and Performance in Immersive Virtual Reality Applications. 1st Joint International Conference on Entertainment Computing and Serious Games (ICEC-JCSG), Nov 2019, Arequipa, Peru. pp.57-69, 10.1007/978-3-030-34644-7_5 . hal-03652010

HAL Id: hal-03652010

https://inria.hal.science/hal-03652010

Submitted on 26 Apr 2022

**HAL** is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
Effects of End-to-end Latency on User
Experience and Performance in Immersive
Virtual Reality Applications

Polona Caserman¹[0000-0002-3252-4533], Michelle Martinussen², and Stefan Göbel¹

Multimedia Communications Lab - KOM, Technische Universität Darmstadt,
Darmstadt, Germany

¹ {firstname.lastname}@kom.tu-darmstadt.de

² {firstname.lastname}@stud.tu-darmstadt.de

**Abstract.** Immersive virtual reality (IVR) offers an opportunity to im-
merge oneself into a virtual world and experience an exciting adventure.
However, latency between a user's movement and visual feedback has a
big impact on user experience and performance. In this paper, we ex-
plore the effect of increased end-to-end latency in IVR applications by
conducting a user study. Firstly, in the searching task, we analyze cy-
bersickness level based on simulator sickness questionnaire. Secondly, in
the reaching task, we measure the user performance by tracking the time
they need to reach a target and the error they make during the execution.
Lastly, in the embodiment task, we measure the sense of body owner-
ship, agency, presence, and latency perception when only one body side
is impaired by latency. We apply the Friedman test with Conover's test
of multiple comparisons as a post-hoc test on all dependent variables to
find significant results. Results show that the end-to-end latency above
63 ms induces significant cybersickness symptoms. In addition, user per-
formance decreases with increasing delay and with end-to-end latency
above 69 ms, the users need significantly longer to complete the task.
Results also show that end-to-end latency affects body ownership signif-
icant later, namely, not until 101 ms.

**Keywords:** Immersive Virtual Reality · End-to-end Latency · Cyber-
sickness · User Experience · Simultaneity Perception · Body Ownership
· Sense of Agency · Sense of Presence · Full-body motion reconstruction

# 1 Introduction

Many users experience adverse effects such as cybersickness when wearing a
Head-Mounted-Display (HMD). The reasons for these symptoms are the sub-
ject of research. One possible theory is the mismatch between the senses which
provide information about the body's orientation and motion, also known as *the
sensory conflict theory* [13]. Thus, increased cybersickness is often reported when
the delay between user input and visual feedback is too high [7,17] or when the
movements are forced upon the users, e.g., in flight simulations [16]. Shafer et al. [15] argue that with increasing sensory conflict, users will experience a higher level of cybersickness.

High end-to-end latency can induce cybersickness [17]. End-to-end latency in immersive virtual reality (IVR) refers to the time delay between a user's action and when this action is visible on the HMD. High delays also impair user performance and experience. Attig et al. [2] state that the lower limit of 100 ms is too high and suggests to update the guidelines.

In this paper, we want to recommend appropriate end-to-end latency so that the researchers and developers can prevent or minimize cybersickness, maximize user performance, and ensure immersive user experience. Our main contributions in this paper are:

- We aim to identify the lower limit for the end-to-end latency in IVR at which the users report increased symptoms for cybersickness.

- We analyze the impact of end-to-end latency on the sense of body ownership, agency, and presence. It should be investigated which maximal delay the users tolerate without losing the joy of the IVR application.

The paper is structured as follows. In Section 2, we present selected related work. In Section 3, we describe the IVR tasks to measure the effects of increased end-to-end latency on user experience as well as performance and furthermore propose our hypotheses. We describe the evaluation method in Section 4. We show the results in Section 5 and discuss them in Section 6. Lastly, we conclude in Section 7.

## 2 Related Work

The work of Albert et al. [1] reveals that participants, even if instructed to search for artifacts, could not find them if the latency was not higher than 50 to 70 ms. Waltemate et al. [18] stated that motor performance and simultaneity perception are worsened significantly at end-to-end latency above 75 ms, whereas the sense of agency and body ownership only decline at latency higher than 125 ms. However, in their study, the participants performed motor tasks inside a Cave virtual environment and did not use IVE with a HMD. Similarly, Kasahara et al. [6] observe that with an end-to-end latency of 104 ms, the participants lose the sense of body ownership.

Kawamura and Kijima [7] studied the effect of latency on the ability to stand on one foot in IVR. They used a Wii Balance Board to measure the stability of the participants and found that balance decreased with increased latency of already 1 ms. Attig et al. [2] expressed a similar view. The researchers report that the users are indeed able to perceive latencies down to the single millisecond and furthermore state that performance already gets impaired by latencies between 16 to 60 ms.

Increased latency of 450 ms also negatively effects the ability to memorize objects in IVR [12]. Furthermore, Samaraweera et al. [14] investigated the impact
of additional latency applied either to the left or right side of the avatar. The researchers found significant differences between the control (with base latency of 45 ms) and experimental group (end-to-end latency of 200 ms). However, only three out of ten participants noticed the added delay. Users in the experimental group walked slower towards the mirror and tried to compensate for the latency by increasing the step and swing time on the impaired side. Likewise, Meehan et al. [10] argue that lower latency of 50 ms compared to 90 ms corresponds to a higher sense of presence.

Stauffert et al. [17] investigated the impact of latency jitter on cybersickness in IVR. In their study, almost half of the participants did not even notice latency jitter and the ones who did, found it to be annoying. The researchers furthermore found a significant correlation between heart rate and cybersickness. Similarly, Waltemate et al. [18] also conclude that the users did not show cybersickness symptoms, even at end-to-end latency above 350 ms. Other works suggest that delays above 40 ms already evoke cybersickness and postural instability [3].

# 3 Experimental Design

## 3.1 Tasks

We want to measure and compare the impact of different latency conditions on user experience and performance. In general, we apply latency to the virtual body and the HMD, controllers, as well as trackers. In other words, player movements are delayed with a certain amount of time. We analyze the impact of tracing delay in IVR to determine the upper latency limit that is still tolerated by users. Therefore, we developed three tasks:

**Searching task** is designed to measure cybersickness through the simulator sickness questionnaire (SSQ) [8]. As proposed by Stauffert et al. [17], the user has to find circular targets (platforms), spawned on the ground in one of three corridors. As soon as the user steps on one platform, a new one randomly spawns in another corridor. Thus, the user has to walk to find the next platform. Only one platform is present in the scene at the same time. The user is exposed to different artificial added delays, between 0 and 100 ms. In each round, the user has to find three platforms, one after another. After that, the user has to state how affected he/she is by each symptom on a four-point Likert scale: "None," "Slight," "Moderate," and "Severe." The scale is converted into integer numbers from 0 to 3. Finally, we calculate the final SSQ scale.

**Reaching task** is designed to measure user performance, in particular, how much time a user needs to touch a cube appearing in a virtual scene and how many errors he/she makes during the execution. Similar to the searching task, we add delays between 0 and 100 ms. As proposed by Ware et al. [19], targets are spawned around the user in the field of view, at a distance so that he/she can reach them. The user has to sit on a stool and should use his/her dominant hand to reach the virtual target. As soon as their hand reaches
the target, the virtual object disappears and a new one spawns randomly in the virtual environment. Thus, the users should reach the targets as fast as possible; however, they must avoid making mistakes. The user can access the object only once. If he/she fails, it is considered as an error.

**Embodiment task** is designed to measure user experience while the users can observe themselves in a virtual mirror. As proposed by Samaraweera et al. [14], we apply latency to only one side of the body (either the left leg and left arm or the right leg and right arm). By using a seven-point Likert scale, we measure the sense of body ownership, sense of agency, sense of presence, and latency perception. The users are exposed to different artificial added delays, between 0 and 200 ms. The questionnaire consists of five questions, as listed in Table 1.

<table><caption>Table 1: Embodiment Questionnaire. Each question was assessed on a seven-point Likert scale: “Strongly Disagree” (–3) and “Strongly Agree” (3).</caption><thead><tr><td>Condition</td><td>Question</td></tr></thead><tbody><tr><td>Body Ownership</td><td>It felt as if the virtual body in the mirror was my body.</td></tr><tr><td>Agency</td><td>My movements caused the movements of the virtual body in the mirror.</td></tr><tr><td>Presence</td><td>I felt as if I was really there.</td></tr><tr><td>Impaired Side</td><td>I noticed a difference in how the avatar responded to my movements between my left and right side. (If the participant answered the question with a value greater than 0, we furthermore asked which side was delayed. The participant could respond with “left,” “right side” or “not sure.”)</td></tr><tr><td>Simultaneity</td><td>The motion of the avatar was simultaneous to my movement.</td></tr></tbody></table>

## 3.2 Hypotheses

Studies on latency in IVR show that users performance may drop and they may notice latency smaller than 75 ms (see Section 2). However, studies evaluating the sense of agency and body ownership show that the users in IVR tolerate higher end-to-end latency between 90 and 200 ms.

For our experiment, we pose our hypotheses as follows:

**H1:** Higher end-to-end latency evokes cybersickness.

**H2:** Higher end-to-end latency decreases user performance.

**H3:** Higher end-to-end latency decreases user experience, in particular, sense of embodiment.

The evaluation of H1 is based on the SSQ, as proposed by Kennedy et al. [8]. Another way to detect cybersickness symptoms could be by using an ECG to measure the electrical activity of the heartbeat as proposed in our previous
work [5]. To test H2, we measure the time it takes for a user to reach a cube and the errors that the user makes. Furthermore, to test H3, we analyze an embodiment questionnaire on a seven-point Likert scale.

# 4 Method

We recruited 21 participants, aged between 19 to 53 years ($M = 28.6$ years, $SD = 8.79$, six females). At the beginning of the experiment, all participants filled out a demographics questionnaire and were informed about the risks.

The experiment was conducted on a laptop with an Intel i7-7700HQ processor and an Nvidia GeForce GTX 1080. This setup fulfills the minimal hardware requirements of the HTC Vive Pro.<fnref n="1" /> The tracking area was approximately 3.1 × 2.9 m large. The application was built with Unity 3D on a Windows 10 system. We used an HTC Vive Pro HMD with two controllers and three trackers to enable full-body motion reconstruction. The users had to bind two trackers to left and right ankle as well as one on the hip. Additionally, they had to hold both controllers in the hands to track arm movements. One additional tracker was bound to a stool so that it can be shown in the IVR application.

Each participant could choose between two virtual bodies (a female and male avatar). After the avatar calibration, the participants then viewed a simple scene with a mirror in front of them and a stool, as shown in Figure 1. Thus, they could view their avatar from the first-person view, i.e., by looking down towards their body or by observing themselves in a virtual mirror.

<figure><img src="image_2.png" /><figcaption>Fig. 1: The virtual environment, used for our experiment. (a) The search task consists of three corridors, where the users have to find platforms, spawned on the ground in one of three corridors. (b) In the reaching tasks, the users sit on a stool and have to touch virtual targets, spawned around them. (c) In the embodiment task, the users have to observe themselves in a virtual mirror, to find the impaired side. Note that the player can see the virtual body from the first-person view.</figcaption></figure>

<footnote n="1">¹ https://www.vive.com/us/support/vive-pro-hmd/category_howto/what-are-the-system-requirements.html</footnote>
The study had a fully within-subjects design. Thus, all participants tested all tasks and all conditions. To ensure that the participants do not adapt to the latency during execution, we randomized the task order as well as latency conditions. The experiment started with a SSQ to provide a baseline measurement. After that, a total of 165 (11 × 3 different platform positions in the searching task, 11 × 6 different cube positions in the reaching task; 11 × 6 tries in embodiment task) randomly conditions were presented. In searching and reaching task, we added a delay between 0 and 100 ms (0 ms, 4 ms, 8 ms, 13 ms, 19 ms, 25 ms, 33 ms, 42 ms, 54 ms, 71 ms, 100 ms). In the embodiment task, we added higher delays between 0 and 200 ms (0 ms, 8 ms, 17 ms, 27 ms, 38 ms, 51 ms, 66 ms, 84 ms, 108 ms, 142 ms, 200 ms).

At the end of each latency condition in the searching task, the participants had to complete the SSQ. The surveyor read the questions and noted the participant's responses. In the reaching task, the participant did not need to complete any questionnaire. Instead, we measured how long each participant needed to grab the cube and how many errors he/she made. In the embodiment task, the participants again had to complete a questionnaire (see Table 1). Note that the order of the tasks and different latency conditions were randomly selected for each participant. Exposure time was approximately 55 minutes.

# 5 Results

## 5.1 Base End-to-end Latency

We firstly measured the system latency of our setup by using a frame counting approach, as proposed by Friston and Steed [4]. The end-to-end latency is the delay between the player's movement until the corresponding avatar movement is displayed on the HMD. Due to the calculation of inverse kinematics for full-body motion reconstruction, our base latency is slightly higher than those found in related work, which is often between 22 and 36 ms [1, 11, 17]. Other similar studies using full-body motion capturing reported higher end-to-end latencies, between 45 and 80 ms [6, 9, 14, 18]. We measured end-to-end system latency of approximately 50 ms.

## 5.2 Cybersickness

To evaluate the level of cybersickness, we calculate the final SSQ scores at the end of each sub-task. Figure 2 shows the final SSQ scores for different end-to-end latency conditions. Error bars represent 95% confidence intervals of the mean.

Results of the Friedman test show that higher end-to-end latency has a significant effect on final SSQ scores ($\chi^2(2) = 100.6, p \le .001$, Kendall's W = .57). The internal reliability for cybersickness was very high (Cronbach's $\alpha$ = .922). Post-hoc analysis with Conover's test and $\alpha$ = .05 reveals significant differences between 50 ms and $\ge$ 75 ms, 54 ms and $\ge$ 75 ms, 58 ms and $\ge$ 75 ms, 63 ms and $\ge$ 83 ms, 69 ms and $\ge$ 75 ms, 75 ms and $\ge$ 121 ms, 83 ms and 150 ms, 92 ms and
<figure><img src="image_3.png" /><figcaption>Fig. 2: Final SSQ scores at different end-to-end latency conditions. Error bars show 95% confidence interval of the mean. The values above the Friedman threshold contribute to the significant cybersickness symptoms. Indeed, values above Stanney et al. threshold already indicate too high SSQ scores.</figcaption></figure>

150 ms, as well as 104 ms and 150 ms. Thus, the results show that higher latency values tend to correspond to higher final SSQ scores.

The results furthermore show significant lower SSQ score before the users put on the HMD (0 ms) and after they experienced end-to-end latency of 75 ms ($p < .001$). Thus, these findings indicate that users will experience sufficient discomfort at end-to-end latency above 75 ms. As it can be seen in Table 2, the reported final SSQ score at 75 ms is already > 22. Stanney et al. [16] also complement these findings, stating that the final SSQ scores of > 20 indicate that such a system is troublesome.

<table><caption>Table 2: Final SSQ scores prior to the users put on the HMD (0 ms) and after each end-to-end latency condition (50 - 150 ms).</caption><thead><tr><th>End-to-end Latency [ms]</th><th>0</th><th>50</th><th>54</th><th>58</th><th>63</th><th>69</th><th>75</th><th>83</th><th>92</th><th>104</th><th>121</th><th>150</th></tr></thead><tbody><tr><th>Mean [ms]</th><td>11.4</td><td>11.8</td><td>14.6</td><td>12.3</td><td>17.1</td><td>16</td><td>22.1</td><td>30.1</td><td>26.9</td><td>29.4</td><td>35.1</td><td>42.9</td></tr><tr><th>SD [ms]</th><td>13.8</td><td>11.2</td><td>13.9</td><td>13.7</td><td>15</td><td>18.2</td><td>18.2</td><td>20.2</td><td>21.3</td><td>22.1</td><td>22.4</td><td>25.7</td></tr></tbody></table>

Moreover, Stanney et al. [16] report that users who report symptoms which are already higher than 15 on the SSQ are experiencing sufficient cybersickness. Comparing this value with the measurement results, the users reported increased symptoms at end-to-end latency above 63 ms (see Table 2). Thus, to minimize cybersickness symptoms, end-to-end latency of below 58 ms (< 15 on SSQ) is required.
## 5.3 User Performance Results

To evaluate user performance, we measure the time the users need to complete the task successfully. Figure 3 shows the average time needed to reach a cube while users were exposed to different end-to-end latencies. Error bars represent 95% confidence intervals of the mean.

<figure><img src="image_4.png" /><figcaption>Fig. 3: Mean needed time for different end-to-end latency conditions. Error bars show 95% confidence interval of the mean. At values above the Friedman threshold, users need significantly longer to complete the reaching task.</figcaption></figure>

Results of the Friedman test show that higher end-to-end latency has a significant effect on time, needed to complete the task ($\chi^2(2) = 68.71, p \le .001$, Kendall's W = .658). The internal reliability of the scale was high (Cronbach's $\alpha$ = .872.) Post-hoc analysis with Conover's test and $\alpha$ = .05 results in significant differences between 50 ms and $\ge$ 69 ms, 54 ms and $\ge$ 92 ms, 58 ms and $\ge$ 92 ms, 63 ms and $\ge$ 104 ms, 69 ms and $\ge$ 121 ms, 75 ms and $\ge$ 121 ms, 83 ms and $\ge$ 104 ms, 92 ms and $\ge$ 121 ms, and 104 ms and 150 ms. In other words, with the minimum end-to-end latency (i.e., 50 ms), users could complete the task faster than with added delay. Thus, the results show that higher latency values tend to require more time to complete the task.

<table><caption>Table 3: Mean needed time for each end-to-end latency condition.</caption><thead><tr><th>End-to-end Latency [ms]</th><th>50</th><th>54</th><th>58</th><th>63</th><th>69</th><th>75</th><th>83</th><th>92</th><th>104</th><th>121</th><th>150</th></tr></thead><tbody><tr><th>Mean [ms]</th><td>16.5</td><td>16.6</td><td>16.9</td><td>17</td><td>17.3</td><td>16.9</td><td>16.9</td><td>17</td><td>17.2</td><td>17.5</td><td>17.7</td></tr><tr><th>SD [ms]</th><td>.7</td><td>.7</td><td>1.5</td><td>1.9</td><td>1.9</td><td>.8</td><td>.9</td><td>1.2</td><td>.9</td><td>1.3</td><td>1.1</td></tr></tbody></table>

By observing Figure 3, we can see that higher delay worsen user performance. Looking at the findings from the post-hoc test indicate that user performance
drops at 69 ms. The data in Table 3 shows that mean needed time values at end-to-end latency of 69 ms and 104 ms are nearly the same. Thus, user performance gets worse with increased delay and most users need significant longer to fulfill the task at end-to-end latency above 69 ms.

Even though the users were allowed to touch the virtual cubes only once, we could not identify any mistakes. We believe that the reaching task is very simple to complete and that more complex tasks would lead to higher chances of performance errors.

## 5.4 Embodiment Results

We conducted a user study to furthermore measure the sense of presence, agency, and body ownership at different end-to-end latency conditions. We calculated the median (MED) and Inter-Quartile Range (IQR) for each question, as it can be seen in Table 4 and Figure 4. Each question was assessed on a seven-point Likert scale: -3 for “Strongly Disagree” and 3 for “Strongly Agree.”

The internal reliabilities for embodiment measurements are high. Cronbach's α for embodiment ranges between .706 (impaired side), .869 (simultaneity), .953 (sense of agency), .988 (sense of presence), and .992 (sense of body ownership). For the sense of agency (MED = 3) and for the sense of presence (MED = 2), we could not find any significant differences. These findings indicate that end-to-end latency of up to 250 ms does not cause a significantly lower sense of agency or presence.

However, results also show that end-to-end latency above 101 ms will decrease the sense of body ownership (see Figure 4a). Indeed, body ownership significant decreases for end-to-end latency above 192 ms. Thus, the sense of body ownership decreases with increasing end-to-end latency. Furthermore, with end-to-end latency above 67 ms, one user already noticed different latencies between the left and right side and also correctly specified the impaired side. As expected, with higher end-to-end latency the users not only perceived increased latency but also correctly specified the side (either left or right body side) which was influenced by additional delay (see Figure 4b). At end-to-end latency of 250 ms, almost half of the users (n = 10) perceived the delay and could correctly specify the impaired side. Similarly, with an end-to-end latency of 116 ms and ≥ 158 ms, the users perceived the movements of the avatar not simultaneously to their movements.

# 6 Discussion

With the significant results of the SSQ comparing pre- and post-conditions, we accept hypothesis H1. Thus, increased end-to-end latency will evoke cybersickness symptoms. Stauffert et al. [17] expressed a similar view. The researchers evaluated latency jitter and found that the disorientation sub-scale was higher in the affected group, although nearly half of the users did not notice the repeated lag in the tracking of the HMD. In contrast to Waltemate et al. [18], we found
<table><caption>Table 4: The table depicts the median (MED) answers for embodiment questionnaire for each end-to-end latency condition. Inter-Quartile Range (IQR) values furthermore indicate the variability of the questionnaire responses.</caption><thead><tr><th>End-to-end Latency [ms]</th><th>50</th><th>58</th><th>67</th><th>77</th><th>88</th><th>101</th><th>116</th><th>137</th><th>158</th><th>192</th><th>250</th></tr></thead><tbody><tr><td>MED<sub>agency</sub></td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td></tr><tr><td>IQR<sub>agency</sub></td><td>1</td><td>1</td><td>1</td><td>1</td><td>1</td><td>1</td><td>1</td><td>1</td><td>1</td><td>1</td><td>1</td></tr><tr><td>MED<sub>presence</sub></td><td>2</td><td>2</td><td>2</td><td>2</td><td>2</td><td>2</td><td>2</td><td>2</td><td>2</td><td>2</td><td>2</td></tr><tr><td>IQR<sub>presence</sub></td><td>2.5</td><td>2</td><td>2</td><td>2.5</td><td>2</td><td>2</td><td>2</td><td>2.5</td><td>2</td><td>2.5</td><td>4</td></tr><tr><td>MED<sub>body ownership</sub></td><td>2</td><td>2</td><td>2</td><td>2</td><td>2</td><td>1</td><td>1</td><td>1</td><td>1</td><td>1</td><td>0</td></tr><tr><td>IQR<sub>body ownership</sub></td><td>2.5</td><td>3</td><td>3.5</td><td>3</td><td>3</td><td>3</td><td>3.5</td><td>4</td><td>4</td><td>3</td><td>3</td></tr><tr><td>MED<sub>simultaneity</sub></td><td>2</td><td>2</td><td>2</td><td>2</td><td>2</td><td>2</td><td>1</td><td>2</td><td>1</td><td>0</td><td>0</td></tr><tr><td>IQR<sub>simultaneity</sub></td><td>1</td><td>1</td><td>1</td><td>1</td><td>1</td><td>1</td><td>1.5</td><td>0</td><td>1</td><td>2</td><td>1</td></tr><tr><td>MED<sub>impaired side</sub></td><td>-3</td><td>-3</td><td>-3</td><td>-3</td><td>-3</td><td>-2</td><td>-3</td><td>-2</td><td>0</td><td>-2</td><td>2</td></tr><tr><td>IQR<sub>impaired side</sub></td><td>1.5</td><td>1</td><td>1</td><td>1.5</td><td>1</td><td>2.5</td><td>1.5</td><td>3</td><td>4</td><td>5.5</td><td>5.5</td></tr></tbody></table>

<figure><img src="image_5.png" /><figcaption>(a) Sense of body ownership, agency, presence, simultaneity, and impaired side.</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>(b) Number of users who identified the impaired side correctly, wrong or were unsure.</figcaption></figure>

<caption>Fig. 4: Median responses for the embodiment questionnaire at different end-to-end latency conditions.</caption>
significant results at the end-to-end latency of 75 ms. However, as suggested by Stanney et al. [16], we want to satisfy the desire of keeping the value beneath 15 on the SSQ. Therefore, we recommend that the researchers and developers should keep end-to-end latency well below 58 ms (see Section 5.2). Our guidelines also confirm those proposed by Attig et al. [2], according to which the lower limit of 100 ms seems to be outdated. Moreover, we indeed recommend keeping the end-to-end latency below 20 ms as suggested by DiZio and Lackner [3].

Furthermore, with the significant results for user performance, we also accept hypothesis H2. Thus, with higher end-to-end latency, the user performance will decrease. In particular, users will need more time to complete the task. Compared to Waltemate et al. [18], stating that motor performance is affected by latency above 75 ms, we obtain similar results. The results show that the higher delays worsen user performance (see Section 5.3). Although the users typically need more time to complete the reaching task with increased end-to-end latency, we could not identify significant differences until 69 ms.

Hypothesis H3 cannot be accepted. We found no significant results for the sense of agency and presence. However, with increased end-to-end latency, body ownership and simultaneity perception will decrease. As already noted by Waltemate et al. [18], the delay affects body ownership significantly later, between 125 ms and 210 ms. Our results show that the sense of body ownership decreases earlier, already at 101 ms; however, we could not find significant differences until 192 ms. Similarly, Waltemate et al. [18] stated that simultaneity perception was affected by latency above 75 ms. Our results show that simultaneity perception dropped at latency values higher than 116 ms.

In contrast to work from Kondo et al. [9], not all our participants were able to identify the impaired side even not in the 250 ms end-to-end latency condition. Nevertheless, with the increased delay, more and more users identified the impaired side correctly. With end-to-end latency of 67 ms only one user and with an end-to-end latency of 250 ms total of ten users were able to specify the impaired side correctly.

# 7 Conclusion

This study aimed to gain insight into the effect of latency on user experience and performance in IVR by artificially increasing the end-to-end latency. We designed three different tasks to evaluate which impact increased latency has in IVR. For quantitative results, we used the Friedman test with Conover's test of multiple comparisons as a post-hoc test. We found significant differences when comparing final SSQ scores between several end-to-end latency conditions. Increased latency negatively affects user experience, e.g., users reported significant cybersickness symptoms with end-to-end latency above 63 ms. Thus, to ensure final SSQ scores of < 15 and minimize cybersickness symptoms, the researchers and developers should ensure end-to-end latency far below 58 ms.

Furthermore, also user performance decreases with higher end-to-end latency. The results show that latency above 69 ms causes significant longer execution
time. The results suggest that the sense of body ownership does accept higher latency values than user performance and cybersickness before significant results can be found. Sense of body ownership decreases with end-to-end latency above 101 ms; however, significant results were not found until 192 ms. We could not find any significant results for the sense of presence and agency. In general, with increasing end-to-end latency, user experience decreases and user performance worsens. We conclude that latency between perceived and physical head movements does not only contribute strongly to cybersickness but also decrease user performance and sense of embodiment.

In future work, we will focus on physiological data, such as ECG to measure the electrical activity of the heart. We believe that this will detect the cybersickness symptoms easier than subjective measurements of a SSQ. Furthermore, the task complexity and duration should be increased and breaks should be included between latency conditions, to avoid carry-over effects.

## References

1. Albert, R., Patney, A., Luebke, D., Kim, J.: Latency Requirements for Foveated Rendering in Virtual Reality. ACM Trans. Appl. Percept. 14(4), 25:1–25:13 (2017)

2. Attig, C., Rauh, N., Franke, T., Krems, J.F.: System Latency Guidelines Then and Now – Is Zero Latency Really Considered Necessary? In: Harris, D. (ed.) Engineering Psychology and Cognitive Ergonomics: Cognition and Design. pp. 3–14. Springer International Publishing, Cham (2017)

3. DiZio, P., Lackner, J.R.: Motion sickness side effects and aftereffects of immersive virtual environments created with helmet-mounted visual displays. Tech. rep., Ashton Graybiel Spatial Orientation Laboratory, Brandeis University (2000)

4. Friston, S., Steed, A.: Measuring Latency in Virtual Environments. IEEE Transactions on Visualization and Computer Graphics 20(4), 616–625 (2014)

5. Garcia-Agundez, A., Reuter, C., Caserman, P., Konrad, R., Göbel, S.: Identifying Cybersickness through Heart Rate Variability Alterations. International Journal of Virtual Reality 19(1) (2019)

6. Kasahara, S., Konno, K., Owaki, R., Nishi, T., Takeshita, A., Ito, T., Kasuga, S., Ushiba, J.: Malleable Embodiment: Changing Sense of Embodiment by Spatial-Temporal Deformation of Virtual Human Body. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. pp. 6438–6448. ACM, New York, NY, USA (2017)

7. Kawamura, S., Kijima, R.: Effect of HMD Latency on Human Stability during Quiescent Standing on one Foot. In: 2016 IEEE Symposium on 3D User Interfaces (3DUI). pp. 141–144 (2016)

8. Kennedy, R.S., Lane, N.E., Berbaum, K.S., Lilienthal, M.G.: Simulator Sickness Questionnaire: An Enhanced Method for Quantifying Simulator Sickness. The International Journal of Aviation Psychology 3(3), 203–220 (1993)

9. Kondo, R., Sugimoto, M., Minamizawa, K., Hoshi, T., Inami, M., Kitazaki, M.: Illusory Body Ownership of an Invisible Body Interpolated Between Virtual Hands and Feet via Visual-Motor Synchronicity. Scientific Reports 8(1) (2018)

10. Meehan, M., Razzaque, S., Whitton, M.C., Brooks, F.P.: Effect of Latency on Presence in Stressful Virtual Environments. In: IEEE Virtual Reality, 2003. Proceedings. pp. 141–148 (2003)
11. Niehorster, D.C., Li, L., Lappe, M.: The Accuracy and Precision of Position and Orientation Tracking in the HTC Vive Virtual Reality System for Scientific Research. i-Perception **8**(3), 0-23 (2017)

12. Papadakis, G., Mania, K., Coxon, M., Koutroulis, E.: The Effect of Tracking Delay on Awareness States in Immersive Virtual Environments: An Initial Exploration. In: Proceedings of the 10th International Conference on Virtual Reality Continuum and Its Applications in Industry. pp. 475-482 (2011)

13. Reason, J.T., Brand, J.J.: Motion Sickness. Academic press (1975)

14. Samaraweera, G., Perdomo, A., Quarles, J.: Applying Latency to Half of a Self-Avatar's Body to Change Real Walking pPatterns. In: 2015 IEEE Virtual Reality (VR). pp. 89-96 (2015)

15. Shafer, D.M., Carbonara, C.P., Korpi, M.F.: Factors Affecting Enjoyment of Virtual Reality Games: A Comparison Involving Consumer-Grade Virtual Reality Technology. Games for Health Journal **8**(1), 15-23 (2018)

16. Stanney, K.M., Kennedy, R.S., Drexler, J.M.: Cybersickness is Not Simulator Sickness. Proceedings of the Human Factors and Ergonomics Society Annual Meeting **41**(2), 1138-1142 (1997)

17. Stauffert, J., Niebling, F., Latoschik, M.E.: Effects of Latency Jitter on Simulator Sickness in a Search Task. In: 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). pp. 121-127 (2018)

18. Waltemate, T., Senna, I., Hülsmann, F., Rohde, M., Kopp, S., Ernst, M., Botsch, M.: The Impact of Latency on Perceptual Judgments and Motor Performance in Closed-loop Interaction in Virtual Reality. In: Proceedings of the 22nd ACM Conference on Virtual Reality Software and Technology. pp. 27-35. ACM (2016)

19. Ware, C., Balakrishnan, R.: Reaching for Objects in VR Displays: Lag and Frame Rate. ACM Trans. Comput.-Hum. Interact. **1**(4), 331-356 (1994)