# Telehealth Using PoseNet-Based System for In-Home Rehabilitation
## 3 Proposed System
### System Architecture  
Figure 1 shows the system architecture of the proposed telehealth system. The system consists of three different modules namely, the _account management_ module, the _virtual appointment_ module and the _prescription_ module. These modules are used by the patients, doctors and system administrators. In the proposed system, the admins are responsible for maintaining the account of both patients and doctors. The patient can make virtual appointments with the doctor or perform the rehabilitation exercises, as shown in the bottom part of Figure 1. A patient can first schedule a virtual appointment with their doctor to discuss their physical condition. They will receive an appointment email notification when the doctor confirms the appointment schedule. During the scheduled appointment date and time, the patient will conduct a video call with the doctor. After discussing with the doctor, they can start the rehabilitation exercises (step 1). The proposed system will then calculate the patient's angular movements (step 2) and store them into the database (step 3). After completing the rehabilitation exercises, a skeleton result (angular movement graph) will be generated, enabling the patient (steps 4 and 5) to examine their angular movements through the graph. The doctors can then access the patient's data (step 6) to evaluate and compare the skeleton results on different selected days (step 7) to provide some advice or prescriptions to the patient (step 8). Finally, the patient can check the prescriptions and make adjustments to their subsequent exercises (step 9). The skeleton detection method used to deploy the proposed system is the PoseNet. It is a machine learning model that can be applied for real-time human pose estimation [27; 28]. By running PoseNet with Tensorflow.js library, anyone with camera-equipped devices such as laptops, PCs and smartphones can experience this technology directly from a web browser. The patient simply needs to stand in front of the device camera. The system will then track all the skeleton points of the patient. The PoseNet can detect a single pose or multiple poses from the images or videos within the browser. For our proposed system, only single-pose detection is implemented. On accessing our proposed system, the patient firstly needs to log in to the web-based telehealth system, as shown in Figure 1(a). The patient then needs to select one of their preferred doctors (Figure 1(b)) before performing rehabilitation exercises. Subsequently, the patient can click the _start_ button in Figure 1(c) to perform the rehabilitation exercises. The system will start the 5-s preparation countdown timer before starting the video capture for the rehabilitation exercise. After completing the rehabilitation exercises, the skeleton result is instantly generated for patient and doctor. Figure 1: Proposed telehealth system architecture. Figure 3 shows the skeleton detection processes during the rehabilitation exercise. First, the patient needs to connect his/her laptop to the internet connection to gain access to the web-based system. When the patient starts the rehabilitation session, their live video is recorded through the webcam and passed as an input to the PoseNet model. When the PoseNet model detects a human pose in the video frame, it extracts the keypoints of the patient by using the 2D coordinate (x, y) to indicate each detected body joint from every video frame. The extracted keypoints are used to calculate the distances and angles of both elbows and knees. When the predefined duration ends, the system will stop the recording and store all four angles from both elbows and knees as an array structure into the database. The patient and doctor can directly access the skeleton results. Based on the same set of actions performed by the patient during the few selected days, the doctor can examine the skeleton result by comparing and evaluating the deviation rate of the angular movements to determine the patient's recovery rate.