# Gamification of Upper Limb Rehabilitation in Mixed-Reality Environment
## 4 Event Sequence
### Finger and Wrist Joint Measurement  
The method used to record the ROM values for the flexion and extension of the finger joints (metacarpophalangeal (MCP), proximal interphalangeal (PIP), and distal interphalangeal joints (DIP)) was far less complex than that used to record the ROM values of the upper arm joints. Furthermore, for the wrist joint, the proposed system can calculate the two DoFs--wrist flexion/extension and radial/ulnar deviation--with high precision. When viewing the user's hand, the HoloLens 2 can locate the different finger joints using  
Figure 4: User interface (UI) for users to input upper arm joint information. (**a**) Game objects for reference on hand along with cardinal axes for the Unity environment (**b**) Interactable sliders for users to input body dimensions.  
the SolverHandler script. Figure 5 shows the visual representation of the hand calibration in mixed reality. From the estimated positions of the finger joints--MCP, PIP, and DIP--the ROM can be calculated. The following example clearly displays how the ROM-calculating algorithm functions. Let us say that the algorithm is calculating the index finger's PIP joint. Then it will locate the two-game objects on the index finger, directly next to the joint of interest. In this case, the game objects are initialized at the user's MCP and DIP joints. For the other two cases, the game objects next to the joint of interest are not always other joints of the finger; for example, for the DIP, the two game objects directly next to the joint of interest are the PIP and the finger tip, and, for the MCP, the two game objects directly next to the joint of interest are the PIP and the finger's respective metacarpal bone. Then, the algorithm calculates the individual distances between the user's joint of interest and the two game objects directly next to it, and the distance between the two game objects next to the joint of interest. The law of cosines formula is then used to calculate the flexion/extension ROM of the finger joint. The recording of the wrist's degrees of freedom also utilizes the same script but is a bit more complex than the finger. The first part of the method uses the script to identify different areas of the hand: the wrist and the palm. When this script is initialized, game objects with the script attached will be spawned in the middle of the user's palm and on the user's wrist. With this information, the system calculates the wrist's different DoFs' ROMs.  
However, for the system to produce accurate results when recording the wrist's DoFs' ROM, the forearm must be facing forward, and the upper portion of the arm must be kept in a still manner, which is a potential source of error. Because the cardinal axes are stationary in the gamified environment, if the arm moves in a certain way or is not facing forward, the recording system may incorrectly measure the wrist's DoFs' ROM. However, a solution to this problem was found; it involves an experimental scenario. This solution is a setup that the user can deploy in order to yield accurate results from the wrist ROM algorithms of the system. The setup makes use of a table or other flat, stationary surface for the user to place their elbow upon. The user will then sit next to this surface in a chair and place their elbow upon the surface as they play the levels that target the wrist's DoFs' ROM. For the calculations to be accurate, the user should avoid utilizing their upper arm  
Figure 5: Visual representation of the hand calibration tool in mixed-reality.  
and moving their elbow such that the variations within the data are solely caused by the user's wrist movements.