# Single Camera-Based Remote Physical Therapy: Verification on a Large Video Dataset
## 2 Methods
### Keypoint Confidence Extraction  
The videos were analyzed using the OpenPose [25] algorithm. OpenPose uses a model with 25 keypoints. In the context of performing rehabilitation exercises, the body segments that are part of the appendicular skeleton are very often measured [26]. These segments allow for translational movement of the body through cyclic movements such as walking [27]. Thus, we used 12 points that allow us to determine the positions and \begin{table}
\begin{tabular}{c c c c c c c c c c c c} \hline \hline \multicolumn{2}{c}{**Total (2133)**} & & & & & & & & & & & \\ \hline \multicolumn{2}{c}{Front View (357)} & & & & & & & & & & Side view (749) & \\ \hline Ly & Kn & Si & St & Ly & Kn & Si & St & Ly & Kn & Si & St \\ \hline
75 & 26 & 186 & 70 & 177 & 95 & 165 & 490 & 215 & 92 & 145 & 297 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Number of videos in each category. Subcategories are: lying down (LY); on the knees (Kn); sitting (Si); standing (St). \begin{table}
\begin{tabular}{c c} \hline \hline
**Starting Position** & **Camera View** \\ \hline lying down (Ly) & front view (frontal plane) \\ \hline on the knees (Kn) & 3\% view \\ \hline sitting (Si) & side view (sagittal plane) \\ \hline standing (St) \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Categorization of videos. Figure 2: Orientation of the camera relative to the subject. Camera view options: front, 3\(\%\) and side [24]. orientations of segments of the appendicular skeleton for further analysis. These 12 points are described in Table 3.  
The outputs of OpenPose for each frame are the x,y coordinate values and the detection confidence for each of the 25 model points, please see Figure 3. From this information, we use only the confidence for the twelve points for each frame. This gives us 12 temporal signals for each unique video. OpenPose processes each frame independently without using the time context. Repeated image analysis returns the same results. OpenPose has almost perfect test-retest reliability within device [8]. In our study, we are not dealing with the absolute position of the detected points. To evaluate the quality of detection, we use directly the confidence returned by OpenPose. Thus, we are not evaluating the accuracy of detection, but the detection capability itself. \begin{table}
\begin{tabular}{l l l} \hline \hline \multicolumn{1}{c}{**No.**} & \multicolumn{1}{c}{**OP Name**} & \multicolumn{1}{c}{**The Most Appropriate Name of the Anatomical Point**} \\ \hline \multicolumn{3}{l}{Upper body} \\ \hline
2 & RShoulder & R. acromion, end of the clavicle (collar bone)—top of shoulder \\ \hline
3 & RElbow & R. lateral epicondyle of humerus, lateral epicondyle of the humerus. Outside of elbow. \\ \hline
4 & RWrist & R. styloid process of the radius; wrist on thumb side. \\ \hline
5 & LShoulder & R. acromion, end of the clavicle (collar bone)—top of shoulder \\ \hline
6 & LElbow & R. lateral epicondyle of humerus, lateral epicondyle of the humerus. Outside of elbow. \\ \hline
7 & LWrist & L. styloid process of the radius; wrist on thumb side. \\ \hline \multicolumn{3}{l}{Lower body} \\ \hline
9 & RHip & R. femur greater trochanter \\ \hline
10 & RKnee & R. femur lateral epicondyle \\ \hline
11 & RAnkle & R. fibula apex of lateral malleelous \\ \hline
12 & LFlip & L. femur greater trochanter \\ \hline
13 & LKnee & L. femur lateral epicondyle \\ \hline
14 & LAnkle & L. fibula apex of lateral malleelous \\ \hline \hline \end{tabular}
\end{table} 
Table 3: The 12 basic keypoints of the appendicular skeleton. Figure 3: OpenPose 25 keypoints model [9]. Detection accuracy using annotated images is a classical metric for comparing machine learning algorithms; accuracy calculations are performed on large image datasets COCO [28], MPII [29]. In contrast, the evaluation of dynamic events has been studied only for single exercises and sub-joints [23]. Existing annotated 2D datasets deal with either images [30] or deal with a small variety of activities [31,32].