## 4 Experimental Setup and Procedures
### Experimental Procedure  
In total, 13 experiments were carried out, each using the experimental setup in a slightly different way. The subject of each experiment, i.e., the evaluated component, and devices involved are listed in Table 2. Each experiment was carried out with and without the corresponding component. The experiments E.6 DF.1 and E.HL.1 were the only ones that did not use NAF. As the MS HoloLens performance was dropping rapidly with the rising number of users, we decided not to perform the E.HL experiments with 15 and 20 client devices (users). The procedure, used in every experiment, consisted of the following steps:  
1. If not already turned on, turn on the server computer SRV. 2. If not already running, launch the web server on SRV, with or without the ECA component. For E.6 DF.1 and E.HL.1, do not launch NAF at all. 3. If ECA is used, launch one instance of the web browser on SRV and open the ECA dashboard page in it. 4. If not already turned on, turn on the desired number of the personal computers PC. 5. On each PC used, launch one instance of the web browser and open the client app with the shared virtual scene in it. 6. If the experiment belongs to the E.6 DF set, turn on the smartphone PH (if not already turned on), launch one instance of the web browser there and open the client app with the shared virtual scene in it. 7. If the experiment belongs to the E.HL set, turn on MS HoloLens device HL (if not already turned on), launch one instance of the web browser there and open the client app with the shared virtual scene in it. 8. Let the client app with the virtual scene run on each client device for a desired period of time. In the case of PH and HL, walk with the device to emulate user activity. 9. After the desired period, close the browsers on the client devices (PC, HL and PH). 10. If a re-configuration is needed, close the web server on SRV. Two performance measures were recorded during the Step 8 of the experiments: frame rate and response time. The measures were recorded on one of the client devices with the constant sampling period of 10 s. For the E.ECA experiments, the recordings took place on one of the personal computers, the E.HL ones on MS HoloLens and E.6 DF  
Figure 7: 3D models used in the experiments: virtual scene (**a**); and user avatar (**b**). on the smartphone. The frame rate was measured in the scene, using the A-Frame API. The response time was measured in the following way:  
1. At the time \(t_{s}\), the client app on the corresponding device (PC, HL or PH) sends a ping packet to the node.js server, running on the SRV. 2. The node.js server responds with a pong packet (32 bytes). 3. The client app receives the pong packet at the time \(t_{r}\). 4. The response time is computed as \(t_{r}-t_{s}\). The ping.js library ([[https://github.com/alfg/ping.js/](https://github.com/alfg/ping.js/)]([https://github.com/alfg/ping.js/](https://github.com/alfg/ping.js/)), accessed on 22 April 2021) was used to implement the measurement. The node.js server is the one that hosts the client app and runs the server side of NAF. In the case of experiments E.6 DF.1 and E.HL.1, only the frame rate was measured. In each experiment, the procedure was repeated twice, first with the evaluated component turned on and second with the component turned off. The time period was set to 60 min for the E.ECA experiments and to 15 min for the E.6 DF and E.HL experiments. The reason for the shorter period was that the E.6 DF and E.HL experiments required human activity and the fact that the shortening had no significant influence on the measured data, thanks to the stable performance of the devices used. As we expect some activity from users in a shared virtual scene, it was necessary to emulate such activity for each client device during the experiments. The activity was represented by a user avatar movement through the scene. In the case of the client apps on PC, the movement was carried out automatically, following the trajectory drawn in green in Figure 8a.