# Impact of Latency on QoE, Performance, and Collaboration in Interactive Multi-User Virtual Reality
## 4 Results and Discussion
### Independent Variables  
Two objective network variables are controlled during the study: latency and burst. Latency, on the one hand, introduces an artificial end-to-end delay between the sending and the delivery of each network packet. This results in delayed synchronization between the actions of one user and the observation of these actions by their collaborator. Values of 0 ms and 500 ms are considered in this study. Burst, on the other hand, introduces a certain probability of blocking traffic for a given time frame, after which all data are sent in a single batch. As such, burst network traffic is emulated. In this study, the burst probability is fixed at 50%, meaning that every packet has a 50% chance of being stalled for a pre-configured time interval. The time intervals being considered are 0 ms and 500 ms. The value of 500 ms was determined based on the works of [PERSON] et al. [4] and [PERSON] et al. [5] [PERSON] et al. [4], on the one hand, explored the effects of network latency on user experience in a multiplayer VR game. An average MOS above 4 (on a 1-5 scale) was obtained on addition of 300 ms of latency. In addition, 80% of users indicated they were willing to continue playing with a delay of 300 ms in place. As such, this shows that higher latency values are required to really challenge user QoE in multi-user VR. [PERSON] et al. [5], on the other hand, performed a similar study on the effects of latency in a multi-user VR exergame. Their results showed QoE scores dropped to an \begin{table}
\begin{tabular}{c l l} \hline \hline
**Session** & **Playthrough 1** & **Playthrough 2** & **Playthrough 3** & **Playthrough 4** \\ \hline
1 & A & B & D & C \\
2 & B & C & A & D \\
3 & C & D & B & A \\
4 & D & A & C & B \\ \hline
5, 6... & \multicolumn{3}{c}{Repeat order} \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Overview of the adopted Latin square ordering of testing conditions for subsequent user evaluation sessions to minimize learning and novelty bias. From session 5 onward, the first 4 orderings are repeated. \begin{table}
\begin{tabular}{c l} \hline \hline
**Timing** & **Content** \\ \hline Pre-session & \(\bullet\) Demographics (age, gender...) \\  & \(\bullet\) Prior experience with VR (Never-Once-Quite-Very) \\  & \(\bullet\) Self-assessed technological proficiency (Low-Medium-High) \\  & \(\bullet\) Baseline VRSQ [26] \\  & \(\bullet\) Ishihara tests [27] \\ \hline In-session & \(\bullet\) Estimation of playthrough time (in sec.) \\  & \(\bullet\) VRSQ [26] \\  & \(\bullet\) Perceived latency, jerkiness + perceived influence on object interaction, \\  & \(\bullet\) collaboration (5-point Likert scale) \\  & \(\bullet\) Perceived spatial and temporal synchrony between collaborators (5-point \\  & Likert scale) \\  & \(\bullet\) Task difficulty (5-point Likert scale) \\ \hline Post-session & \(\bullet\) Rank network configurations based on perceived network optimality \\  & \(\bullet\) Rank network configurations based on perceived cybersickness inducement \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Overview of the timing and content of the subjective evaluations during a single experimental session. average MOS of 2.5 when the user's personal latency reached 500 ms; this shows that user experience drops sharply within small intervals of increasing latency. Furthermore, end-to-end latencies of 400-600 ms have been shown to occur in real scenarios [24]. Therefore, in order not to surpass the edge on playability while remaining consistent with possible real-life occurrences, we decided not to push this further and to identify 500 ms of latency as an interesting value that sufficiently challenges the end-user QoE. The 50% blocking change was empirically determined to optimize the trade-off between providing a distortion in the system that was strong enough to be observable by human subjects while still allowing for sufficient playability of the experience in order to collect relevant results.