# On the Design and Implementation of a Mixed-Platform Collaborative Virtual RAS Planner
## 4 Experimental Study  
An experiment was conducted to validate the system for clinical practice at Hospital General de Valencia, Spain. For this purpose, four domain experts were recruited, three men and one woman. All of them were experienced surgeons of this hospital, with more than two years of experience performing surgical procedures with a Da Vinci X surgical robot [30]. Informed consent was obtained from all four experts. For the assessment process, two devices were used: a Microsoft HoloLens 2, as an example of an XR device, and a Meta Quest 2, as an example of an immersive VR device. All four experts used both devices. A third person, not included in the assessment process, was also present in the virtual space using a laptop, in order to add complexity to the collaborative experience and project this third view onto a big screen so that all external observers were able to see the use of the tool. Finally, another PC was used as a server to host Photon. Figure 10 shows a picture of the experimental validation, in which we presented to the surgeons a virtual patient with a complex tumour in the abdominal wall  
Figure 8: Two different snapshots of the console view of the virtual trocar tool after activating the virtual robotic arms. A simulated version of the console of the robot is utilized. The view also includes the robot arms, which can be moved to check the dexterity and workspace of the surgical tools. Figure 9: Meeting functions, with which each user can share their point of view and use that of others. that infiltrated adjacent tissue, in order for them to assess the use of the collaborative virtual RAS planner. The procedure followed during the experimental session consisted of the following steps:  
1. Short briefing about the interaction of each platform and the use of the tools. 2. Free collaborative session with a 3D model segmented from a CT scan of the abdomen, where the following steps were conducted: 1. First, experts #1 and #2 tested the system simultaneously for 15 min, one with the XR platform and the other with the VR platform. 2. Then, experts #1 and #2 exchanged platforms and repeated the testing. 3. Afterwards, experts #3 and #4 tested the system simultaneously for 15 min, one with the XR platform and the other with the VR platform. 4. Finally, experts #3 and #4 exchanged platforms and repeated the testing. To assess the usability of the tool, the SUS questionnaire was used [31]. In questions SUS-1 to SUS-10, the range 1-5 means: 1: strongly disagree, 5: strongly agree, where odd questions are formulated in a positive way (thus, the best possible score is 5) and even questions are formulated in a negative way (the best possible score is 1). To assess the tools of the application in a more specific way, we also used a satisfaction questionnaire, in which we asked eight specific questions tailored for this application. This questionnaire focuses on the tools provided by the application and on the usefulness of the collaborative use of the virtual planner. Table 1 lists the questions of the satisfaction questionnaire, which were asked in a 7-point Likert scale (1: strongly disagree, 7: strongly agree). Figure 10: Experimental validation. Domain experts also commented on the reasons for their answers. All of them agreed that navigation in VR was \"very easy to use\", because it resembles the use of the surgical robot. They all liked the first-person perspective of the trocar. One participant regarded this option as \"very useful\". They also considered it useful to collaborate with other people in a virtual shared environment, something they did not consider as possible for the planning task. Finally, they also reported that the visualisation of the tissues was \"confusing when the point of view was inside the body\". Overall, the impression of the domain experts is that the application is useful for preparation and discussion of surgical planning with RAS.