# Impact of Latency on QoE, Performance, and Collaboration in Interactive Multi-User Virtual Reality
## 3 Materials and Methods
### Experimental Methodology
#### 3.2.2 Testing Procedure  
Figure 4 shows the multiple steps of the testing procedure of one experimental session. Note that two users participate concurrently during a single session, and that participants  
Figure 3: Schematic overview of the multiple steps to be taken during the collaborative task. These are numbered 1–13 in the order they need to be performed as discussed above. Arrows indicate object transformations, e.g., combining water and flour to a ball of dough in (1).  
Figure 2: The virtual environment at the start of the experience.  
are assigned randomly to the pairs. First, participants are welcomed and given a brief introduction on the main purpose and organization of the experiment. In addition, they are presented with a written informed consent form in which they are asked for permission to collect and process their anonymized data and for their personal information, such as age and gender, to be used in the study. Additionally, participants are notified about the possibility of experiencing cybersickness. Furthermore, they are informed on their rights regarding the possibility to withdraw from the study at any given time and on the access to, correction, and deletion of their collected data. Next, participants are asked to complete a pre-session questionnaire, which asks about gender, age, and prior experience with VR, and subjective evaluations in general. They are also asked to assess their own technological proficiency and to fill in a prior baseline Virtual Reality Sickness Questionnaire (VRSQ) (developed and validated by [PERSON] et al. [26]). In addition, the participants are tested for correct color vision using the Ishihara tests [27]. Afterwards, subjects are given a brief oral instruction session on the use of VR (headset, boundaries, controllers...) and the actual use case. Participants are offered the possibility to ask questions in case anything is unclear, which are answered accordingly.  
Then, the actual experimental session takes place following a within-subjects design. It consists of four consecutive playthroughs, where each round consists of a different network condition (A to D). The order of these testing rounds for each session is determined using a balanced Latin square design to avoid any bias due to ordering and learning effects (Table 3). Note that participants are intentionally kept unaware of the particular scenario configurations and ordering. After each round, participants are requested to remove the headset and fill in the in-session questions. Here, participants are asked to give an estimation of the playthrough time in seconds, as well as to fill in an additional VRSQ to estimate the influence of the network configuration on inducing cybersickness effects. Furthermore, they are asked to what extent they noticed any latency or jerkiness (the observable result of burst traffic in a networked VR system), as well as to what extent these interfered with physical object interaction in the system and collaboration between participants. Participants are also asked to rate the assumed spatial and temporal synchrony between their own view and the view of the collaborator and to rate the difficulty of the task given the current network conditions. All these questions are rated on a 5-point Likert scale.  
After the four experimental rounds, subjects were requested to fill in a short post-session questionnaire as well. Here, subjects were asked to rank the four playthroughs based on level of sickness inducement and optimality of the network conditions. Table 4 provides an overview of the different subjective evaluations and their timing within the experimental session. Once the post-session questionnaire was finished, subjects were thanked for their participation before closing the session.  
Figure 4: Schematic overview of the experimental flow of a single experimental session.