# Multimodal Fall Detection Using Spatial-Temporal Attention and Bi-LSTM-Based Feature Fusion
## 2 Related Work
### Video-Based Fall Detection System  
Typically, human silhouettes are segmented from videos captured by RGB cameras or Kinect, and features are extracted for fall detection. Common approaches are based on traditional ML, such as SVM and CNN [28, 29, 30] models. Fall detection has been proposed using the ratio of the height and base of the triangle formed by the head and two feet [31]. [PERSON] et al. proposed a two-stream approach using MobileVGG [32]. This model adopts human motion features. Although such ML algorithms can achieve good accuracy in some cases, it is difficult to achieve it efficiently on large-scale and dynamic video datasets. The recent development of DL techniques can also be used in fall detection. [PERSON] et al. directly applied a convolutional neural network (CNN) to each frame image in the video to perform fall detection [33]. [PERSON] et al. proposed a fall detection method based on a three-dimensional (3-D) CNN-based fall detection method [34]. Three-dimensional CNN can extract motion features of time sequences that are important for falling detection and combine three-dimensional CNN with LSTM. Fall detection has been proposed employing a multi-stream CNN based on a multi-level image fusion approach [35]. It determines the difference in motion within 16 frames of the input video. It achieved an accuracy of 99.03% on the Le2i dataset. Traditional image and video systems have difficulty in effectively separating the foreground from the unwanted background, which can reduce the effectiveness of the entire system. In addition, the enormous amount of computation requires a lot of energy and memory. Recently, more and more researchers have adopted skeleton-based systems for this task. In skeleton-based approaches, a person's pose is estimated from an RGB image, and features are extracted from the skeletal joints [36; 37; 38; 39; 40; 41]. [PERSON] et al. proposed a transformer model that estimates the pose of a person on a video image and employs transfer learning [40]. [PERSON] et al. adopted the torso angle as a new feature for recognizing falls and adopted SVM to determine whether the action was a fall [37]. Features are calculated by extracting the human skeleton from the depth image, and this method reported 93.56% accuracy with the TSTV2 dataset. Another researcher has worked to develop a real-time fall detection system using a short-time Fourier transform (STFT) and a 1D-CNN, achieving accuracies of 91% on MCFD, 99% on UR Fall, and 98% on NTU RGB + D datasets [40]. Another researcher identified 15 key points from the Alphapose network and divided it into five segments [42]. A new feature descriptor can be generated by extracting the segmented parts' distance, angle, and tilt angle. They achieved an accuracy of 98.32% using the UPfall dataset. A fall detection model has been proposed that combines an SVM and a KNN model based on important spatial features of the foreground image [43]. They select keyframes from the height-to-width ratio displacement and the center of gravity displacement in the horizontal and vertical directions. The proposed fall detection system obtained a maximum peak accuracy of 98.6% and a recall of 100% in detecting falls. Many approaches that employ skeleton data convert joint points into metric data, such as distance and angle. During this conversion process, the original joint position data may be lost. This loss of data creates the risk of missing important information in motion analysis and recognition models. To effectively tackle the challenge above and simultaneously enhance the precision of the outcomes, [PERSON] and his colleagues undertook the development of an innovative DL methodology, which they have designated as the spatio-temporal Graph Convolutional Network (ST-GCN), a significant contribution to the field of artificial intelligence and ML, as referenced in their research [36]. ST-GCN treats skeleton data (joint point positions) as a graph structure.