# Augmenting Industrial Control Rooms with Multimodal Collaborative Interaction Techniques
## 4 The Study
### Experiments
#### 4.1.3 Synchronous Collaborative Planning  
To test RQ 3, we created a collaborative prototype application featuring a reasonable use case in the context of our project. The idea is that one worker in the production hall updates the task planning together with one planner from the office. The application view of the desktop user is shown in Figure 7.  
The interaction surface of the user interface is visually the same for the desktop and HoloLens user, which consists of two slates: one is the slate \"current planning\" which shows a timeline, where tasks can be added into time slots. The other slate contains the tasks which could not be allocated to a time slot yet. Tasks can be moved around via drag-and-drop by mouse (desktop user) or by hand-ray (HoloLens user). Releasing a task outside of the slates pushes it back to the initial position before dragging.  
To assess RQ 3, we set up two experimental settings: in one (setup 1) we display cues of co-presence for both users, and in the other no cues are visible (setup 2). In setup 1, the desktop user can see the hands of the HoloLens user, as well as the hand-ray, i.e., the extension of the finger pointer that can be used for far-interaction, as well as his gaze focus. The positions of the visual cues are retrieved with the help of the Mixed Reality Toolkit (MRTK) from Microsoft. The HoloLens user can see the mouse pointer of the desktop user.  
The synchronization between HoloLens and desktop app is done with the help of Photon Unity Networking 2, using their free server resources and the Photon.Pun library. Here, a lobby is created on startup from the HoloLens app and all elements are synchronized with their position, rotation and scale for the desktop app. Photon uses an ownership system, where elements created by one user, are usually owned by that user and can only be moved by him or her. To give both users the ability of moving tasks around, remote procedure calls are used.  
A pair of users (collaborators) is needed to conduct this experiment. One user uses the desktop application with the mouse and the other user uses the HoloLens application. The collaborators can communicate with each other by voice chat. As described in the previous section, there are two alternative experimental setups evaluated: in one setting, visual cues of the users' movement are shown to each other, whereas the other setting omits those cues. The cues visible to the desktop user are the hands, the focus point of the view, and the pointer from the hand of the HoloLens user. The HoloLens user can see the position of the mouse on the task planning surfaces. To antagonize the learning effect happening in the first run, potentially skewing the results of the second run, we swap the starting setting between each evaluation. Both users receive independent conditions that  
Figure 7: View of the collaborative application from the perspective of the desktop user.  
they have to satisfy in the task planning task with the form \(<\)Task A\(>\) comes after \(<\)Task B\(>\). If it is not possible to satisfy the conditions for both users, the collaborators have to find an optimal compromise. This means that the deviation of satisfied conditions shall not be more than one. In order to answer RQ 3, we asked the following questions to the Desktop user:  
* _How much did the visualization of the hand of your collaboration partner help you to solve the task?_
* _How much did the visualization of the view of your collaboration partner help you to solve the task?_  
The HoloLens user was asked:  
* _How suitable are Holograms and control via hand gestures for the task?_
* _How much did the visualization of the mouse of your collaboration partner help you to solve the task?_