# A Framework for Fall Detection Based on OpenPose Skeleton and LSTM/GRU Models
## 2 System Architecture and Method
### Data Pre-Processing
#### 2.3.3. Compensation for Missing Value  
OpenPose still uses 2D image features to obtain the skeleton, so when these features are missing, problems such as unpredictability or prediction errors will occur. As shown in Figure 7, when the human body is overlapped, obscured, the outline of the human body is not clear, or the upper and lower feet of the human body appear, it will cause loss and errors when generating the skeleton. In this paper, we use a linear interpolation method to compensate for the missing values. We set 100 pictures as a group and use the original non-missing value data to interpolate missing data in the following pictures to ensure that there is enough data.  
Figure 5: The position and number of the original human skeleton coordinates.  
Figure 6: The position and number of joint points after displacement.  
The interpolation method can estimate the subsequent data value through the original data. If the original number of the joint points is too few, the error of the calculated missing value to its actual value will be large. Linear interpolation uses two known data points and calculate the slope of the two point. The estimated value \(Y\) is defined as:  
\[Y=Y_{0}+\frac{Y_{1}-Y_{0}}{X_{1}-Y_{0}}(X-X_{0}), \tag{8}\]  
where \(X_{0},~{}Y_{0},~{}~{}X_{1}\) and \(Y_{1}\) are known data in the dataset, and \(X\) is any data between \(X_{0}\) and \(X_{1}\). Linear interpolation is fast and simple, but the accuracy is not high. In this paper, we set a threshold such that if the original joint points (\(x_{n}^{f},~{}y_{n}^{f}\)) are missing more than 67%, no compensation will be performed.  
As an example, Figure 8 shows the \(y\) coordinates of a hand joint point of 30 consecutive pictures while a person makes a gesture of raising and lowering the hand. The first 15 pictures are gradually raising the hand, and the hand is lowered from the 16 th picture. We use these data to evaluate the error between the compensated values and the original values.  
Figure 9 shows when the turning point is not a missing point, there will be no excessive error between the estimated value and the actual value. As shown in Table 2, the original dataset has a total of 2,850,000 joint points, and the missing points are 520,112. After normalization, there are 15 joints left. The original dataset has 1,710,000 with 214,707 missing points. The number of joint points after missing value compensation is the same  
Figure 8: The hand y point changes.  
Figure 7: Missing joint points. (**a**) The arm is obscured by the body, resulting in the loss of wrist joints; (**b**) Because the front and back features of the 2D image are not clear, Part Affinity Fields (PAFs) and Confidence Map cannot calculate the human skeleton.  
as that after normalization, while the number of missing points is reduced to 27,849. Among them, the missing joint points of the original dataset accounted for 18.2%. After regularization, 10 unimportant joint points were removed, and the missing joint points accounted for 12.5%. After the missing value compensation, the missing joint points accounted for 1.6%. The reduction in missing points can help the model to reduce noise and increase accuracy during training.