# Gamification of Upper Limb Rehabilitation in Mixed-Reality Environment
## 5 Experiments and Results  
The functionality and usefulness of the proposed rehabilitation system were examined with ten healthy adult volunteers with no past arm-related injuries and two occupational therapists (OT) (_n_ = 12). The average age, height and body weight of the participants were 26.3 \(\pm\) 2.6 years, 175 \(\pm\) 10 cm and 135 \(\pm\) 25 lb, respectively. However, the target population of the proposed system includes patients with Motricity Index (MI) scores (50-100) focusing on pinch grip, and elbow flexion and shoulder abduction muscle force, mobilization of upper limbs. Figure 7 depicts users using the HoloLens 2 to play interactive games. The system could perform as anticipated and accurately record the joints of interest in the arm. Without any glitches in data collecting, the algorithm produced a CSV file with all the information. An informal assessment of the participants' post-control testing showed excellent user experience with minimal to no technical issues. The game's goal was clearly explained to the participants and was well-accepted as entertaining and engaging. A breakdown of the game's difficulty by DoFs and ROM is provided in Table 1.  
Figure 6: Data flow of the system. While the participant was playing the level shown in Figure 2b, in-game text displays upper arm joint ROM, as shown in Figure 8. As the Kinect V2 sensor is proven to be effective [41; 42; 43] in measuring upper-body kinematics, the accuracy of the upper limb joint was determined by comparing arm motion data statistically with the data calculated based on the Kinect sensor data while the users were playing the games [44]. Figures 9 and 10 display the joint angles a user exhibited throughout a session of playing the game displayed in Figure 2a as measured by the proposed system and the joint angles generated by the MATLAB program (based on Kinect sensor data) for the same user, respectively. Figure 8: In-game text displaying the ROM of the upper limb joints’ DoFs while a user is playing the game in Figure 2b. Figure 7: User testing the proposed MR-based video game on the HoloLens 2, (**a**) the user is knocking down structures in a game level (**b**) the user destroying all the spheres in a game level (**c**) completing a game level by picking an apple from a tree. Table 2 shows the average 1-sample \(t\) -test \(p\)-values for all ten control testing individuals to assess MR-game precision and effectiveness. As seen in the table, all of the p-values are above the proposed \(\alpha\) value of 0.05. Due to this, the 1-sample T-test shows that the ROM values recorded by both the MR system and MATLAB program are statistically insignificant of each other, in turn, validating the accuracy of the MR systems capability to correctly record the user's upper arm joints' DoFs' ROMs. Figure 10: The upper limb joint angles as calculated based on the Kinect sensor data during the same session as in Figure 9.  
Figure 9: The upper limb joint angles as calculated by the proposed MR system during a session when one of the subjects was playing the game displayed in Figure 2a. This \"sensorless\" system was achieved by developing custom algorithms for measuring the joints'--of the upper extremity, that is--DoFs' ROMs. This was done using C# scripts, RadialView, and SolverHandler, provided by the MRTK, as explained in Section 2. These two programs located some joints of interest for the proposed system, such as the finger and wrist joints. Then, using mathematical algorithms based on certain geometric principles, the ROMs of the finger and wrist joints were easily found. Recognizing and finding the ROM of other joints served to be more challenging. These particular joints, the shoulder and elbow, required custom programs due to the HoloLens 2 not being able to recognize them upon gazing at those joints. These programs relied heavily on the user's body dimensions to make assumptions about the user's structure and estimate the position of the elbow and shoulder position.