## 2 Proposed System
### Posture and Action Recognition  
Posture recognition is used to determine the readiness of a person for temperature measurement in the fever station, as well as the measurement location on his/her forehead. Meanwhile, it is also utilized to detect whether a patient has actually taken the medicine in the isolation ward according to his/her arm motions. The system initially captures images using the depth camera and subsequently feeds them into the OpenPose algorithm, a renowned method for 2D human skeleton identification and processing [24]. As one example of posture analysis, shown in Figure 2, 25 key joint points on the human body are identified. The system then uses the point cloud information for depth \(Z_{i}\), also obtained from the depth camera, to derive the 3D coordinates (\(X_{i},~{}Y_{i},~{}Z_{i}\)) for point \(i\) from its corresponding 2D coordinates (\(x_{i},~{}y_{i}\)), which are represented in the coordinate system of the camera, as described in (1):  
\[ \begin{bmatrix}X_{i}\\ Y_{i}\\ Z_{i}\end{bmatrix} =Z_{i} \begin{bmatrix}f_{x}&s&c_{x}\\ 0&f_{y}&c_{y}\\ 0&0&1\end{bmatrix} ^{-1} \begin{bmatrix}x_{i}\\ y_{i}\\ 1\end{bmatrix}  \tag{1}\]  
where (\(c_{x},c_{y}\)), (\(f_{x},f_{y}\)), and \(s\) are the center coordinates of the image, focal lengths expressed in pixels, and a scale factor, respectively, and the unit is meters. We use the neck, hip, and knee locations out of the 25 key points to check whether the person has already sat still for temperature measurement, as well as the shoulder, elbow, and wrist for the action of medicine taking. As the recognition of a sitting posture is relatively straightforward, it can be determined by simply inspecting whether the angle between neck-to-hip and hip-to-knee is close to 90\({}^{\circ}\) for a period of time. On the other hand, the checking of medicine-taking action is more challenging, as it demands the analysis of the time sequence corresponding to the entire action, while there may exist slight differences in motion not only for different  
Figure 2: An example of posture analysis involves the assessment of the human body using 25 key joint points. persons, but even for the same one. In response, we propose a novel algorithm, SDTWR, based on the concept of dynamic time warps [19], frequently employed for evaluating the similarity between two temporal sequences of different speeds and found to be effective in applications such as speech and handwriting recognition, as well as motion analysis. The SDTWR operates as follows. First, the time sequence of the medicine-taking action acquired via the depth camera is taken as the input. For each sample on the sequence, the shoulder, elbow, and wrist locations of the arm are grouped together to form a set \(S_{i}\) that represents a posture, expressed as  
\[S_{i}= \begin{bmatrix}J_{i,}&s\\ J_{i,}&e\\ J_{i,}&w\end{bmatrix} = \begin{bmatrix}X_{i,s}&Y_{i,s}&Z_{i,s}\\ X_{i,e}&Y_{i,e}&Z_{i,e}\\ X_{i,w}&Y_{i,w}&Z_{i,w}\end{bmatrix}  \tag{2}\]  
where \(J_{i,}\;s,J_{i,}\;e\), and \(J_{i,}\;w\) represent the 3D coordinates of the shoulder, elbow, and wrist, respectively. For \(S_{i}\) among a total number of \(m\) training sets, it needs to be normalized to become \(S_{N_{i}}\):  
\[S_{N_{i}}=\frac{S_{i}-\underset{1\leq i\leq m}{\text{min}}\{S_{i}\}}{\underset {1\leq i\leq m}{\text{max}}\{S_{i}\}-\underset{1\leq i\leq m}{\text{min}}\{S_ {i}\}} \tag{3}\]  
Consequently, the action of medicine taking can then be viewed as the combination of a series of \(S_{N}\). To achieve a better recognition effect, the entire medicine-taking action (MTA) is further divided into five sub-actions according to evident differences between them, as shown in Figure 3. To acquire the typical MTA as the reference, we invited 10 subjects and asked each of them to conduct the MTA three times. These 30 collected time sequences, pre-processed via (3) to become that of \(S_{N}\), were then sent to the CNN for training and classification.