## 2 The Proposed Action Identification System for Home Upper Extremity Rehabilitation
### Head Skeleton Point Determination  
To rapidly determine the skeleton points of the human head, we assume that when the rehabilitation user is sufficiently near the screen center, then the head skeleton can be detected in the central region of the screen, denoted by \(\mathrm{S_{C}}\). We adopted the skeleton feature map and the detected skin region, denoted by SK and SR, respectively, to locate the head skeleton point. Because the user's face should be sufficiently close to the camera, the area of the face's skin region should be sufficiently large. The search process for the head skeleton point is performed through the following steps. **Step 1:** Scan the skeleton feature map within the circular region whose center is located at the central coordinate of the input image with a radius of max(W/4, H/4), where W and H denote the width and height of the image, respectively. **Step 2:** If the skeleton feature map can be obtained in Step 1, then we validate the corresponding connected-component area of the skin color region covering the skeleton feature map that is sufficiently large to be a human face--its area should be larger than a given threshold \(\mathrm{T_{s}}\), where the threshold is set at 800 pixels in our experiments. As a result, we can obtain the location of the head skeleton point, denoted by \(\mathrm{S_{H}}\) and depicted in Figure 11.  
Figure 11: Establishing the skeleton point of the head: (**a**) The user approaches the preset point, which triggers detection. (**b**) The skeleton feature map combined with YCbCr elliptical skin color detection establishes the head skeleton point. Figure 10: Skin detection process: (**a**) RGB image, (**b**) Elliptical skin model, (**c**) Morphological close, and (**d**) Largest connected object. Shoulder Skeleton Point Determination  
After the skeleton point of the head (S\({}_{\text{H}}\)) is established, we can determine the pair of shoulder skeleton points according to the position of the head because the shoulders should appear under the two sides of the head:  
**Step 3:** To find the initial shoulder skeleton points, we first search down the rectangular boundaries of the box formed by the head's skin color region to locate the pair of skeleton feature points that are first encountered under the head. Then, we set these two skeleton points as the initial left and right shoulder skeleton points, denoted by \(S_{SL}\) and \(S_{SR}\) respectively. **Step 4:** As depicted in Figure 12a,b, the initial shoulder points are possibly determined on the basis of the clavicle positions under the head's boundaries, but the actual shoulder points should be closer to the lateral ends of the clavicles. Therefore, we set the initial shoulder points \(S_{SL}\) and \(S_{SR}\) as the centers of the corresponding semicircular regions formed by the movement regions of the two arms. **Step 5:** Because the real shoulder points should be at the rotational centers of the shoulder joints, we set the initial shoulder points \(S_{SL}\) and \(S_{SR}\) as the centers, and set a radius \(r\) with the angles \(\theta\) ranging from \(90^{\circ}\) to \(-90^{\circ}\) of the semicircular regions of the left and right shoulders. Then, we determined whether skeleton feature points were present over the two semicircular regions through Equation (2). \[ \begin{split} S_{X}=r\times\cos\theta+S_{x}\\ S_{Y}=-r\times\sin\theta+S_{y}\end{split}  \tag{2}\]  
where \(S_{x}\) and \(S_{y}\) denote the \(x\) and \(y\) coordinates of the initial shoulder points (i.e., \(S_{SL}\) and \(S_{SR}\)), respectively, and \(S_{X}\) and \(S_{Y}\) represent the \(x\) and \(y\) coordinates of the possible shoulder points we evaluated along the semicircular regions of the left and right shoulders, respectively. Because the real shoulder point will be the rotational center of the shoulder joint, the candidate shoulder point should accord with this characteristic.