# Producing Synthetic Dataset for Human Fall Detection in AR/VR Environments
## 4 Experiments
### Recognition Results  
Figure 8 presents the simulation results of the digital human in two different rooms, where top--ground truth segmentation maps (hit map = 1.0; floor = 0.79; human = 0.66; walls = 0.48), and bottom--hit map predictions by model with rendered rgb images. In Figure 8a--c, the example images before falling from different cameras and labelled \"did not fall\" were shown, and Figure 8d--f, images are labelled \"fall\". The recorded samples of fall scenarios represent different interactions with the environment during a fall, which can be divided into three main cases: the subject falls forward (Figure 8d), falls back (Figure 8e) and falls on the side (Figure 8f), the description of which will be detailed below.  
It is possible to evaluate the unique scenarios of the fall, and the recorded data allows one to judge the nature of the impact as a result of conducted simulations with digital humans. Finally, all three simulation examples in Figure 8d--f have a head hit registration, as seen from the hit map data from the bottom of the figure.  
As expected, increasing the variance of data generation in the second experiment (2 nd exp) improved the model efficiency at the testing stage, while the amount of data used for  
Figure 8: Ground truth (top) synthetic data generated in fall simulations; hit map predictions for digital human stand and fall positions (bottom). (**a–f**) Minimum detection confidence = 0.9.  
Figure 7: Loss functions for Mask R-CNN model trained on digital human fall data for the first and second simulation experiment.  
training was more than halved. The accuracy of the testing set was calculated considering \"fall\" and \"not fall\" classes detection. The value was 91.4% and 75.8% (1 st exp) compared with 97.6% and 92.1% (2 nd exp). Additionally, the 2 nd exp trained model data appears to be more effective in the hit map prediction. Table 1 shows examples of comparison of specific predicted hit maps training data on the example of three types of falls.  
Hit map recognition on the generated synthetic data shows promising results. It was shown in Table 1 that predicted hit maps are able to convey information about the impact of individual parts of the body, so in all examples a head hit is traced. For comparative analysis, we specify the minimum detection confidence for the predicted mask with a value of 0.9. As a result, for a forward human fall example, we had following DICE metrics of 75.0% (2 nd exp) with a comparison of 71.9% (1 st exp) and IoU metrics of 59.7% (2 nd exp) with a comparison of 55.5% (1 st exp). The results of the segmentation metrics comparison were presented in Table 2.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Fall Type** & **GT** & **Predicted Map, 1 st Exp** & **Predicted Map, 2 nd Exp** \\ \hline Forward fall (Camera \#2) & & & \\ Forward fall (Camera \#11) & & & \\ Back fall (Camera \#9) & & & \\ Back fall (Camera \#1) & & & \\ Back fall (Camera \#1) & & & \\ Back fall (Camera \#4) & & & \\ Side fall (Camera \#1) & & & \\ Side fall (Camera \#6) & & & \\ Side fall (Camera \#1) & & & \\ \hline \hline \end{tabular}
\end{table}
Table 1: Ground truth (GT) and specific predicted hit maps for Mask R-CNN model trained synthetic data.  
In order to ensure that the proposed synthetic data generation pipeline is effective in real-world datasets, we conducted the experiment using the UR Fall dataset (URDFD) [52]. In Table 3, the accuracy for results for fall and not fall detection was given. This experiment shows the reliable efficiency of the proposed approach, and the results are comparable with the recognition accuracy obtained during training on real data presented in [53], where an accuracy of 95% was achieved. Figure 9 provides the recognition results on real data using Mask R-CNN trained by the synthetic data.