## 2 Related Works
### Our Merged Action Dataset: ALMOND  
By gathering samples from the five existing datasets that provide a meaningful contribution to our defined set of actions, we can produce an aggregated dataset that is inherently heterogeneous. The adopted subdivision into three groups of actions implies that no action belonging to a specific group excludes those from a different group. Conversely, actions inside each group are mutually exclusive to each other, i.e., they cannot be performed at the same time. Furthermore, it is fundamental to consider the inclusion of a reject class, in order to contemplate the inference on actions that were never seen during the training of a specific group. The Status group is an exhaustive set, and as such it does not need a reject class (i.e., the subject must necessarily perform one of the involved actions). Conversely, for Alerting and Daily-life we explicitly included a sample of instances from all the other action groups. The underlying idea is to train a model that is robust with respect to collateral movements of a subject performing actions outside the action group. This is necessary for a real-life application, as most of the datasets are performed by a still person, only moving the strictly necessary body parts, and as such depicting an artificial execution of the action. Table 4 shows the number of samples provided by each dataset for each action. As reported, the Status group is composed of the largest variety of datasets, as all datasets give samples for almost all its actions. The remaining groups are markedly bimodal, meaning that for each action the samples \begin{table}
\begin{tabular}{l c c c c c c} \hline \hline
**Action** & **IXMAS [45]** & **UWA3D II [52]** & **N-UCLA [50]** & **MSR 3D [48]** & **NTU [27]** & **Total** \\ \hline
**Status** & & & & & & \\ Seated & & 36 & 54 & 20 & 948 & 1058 \\ Standing & 36 & 36 & 53 & 20 & 948 & 1093 \\ Lying & & & & 20 & & 20 \\ On the floor & 36 & 70 & & & & 106 \\ Walking & 36 & 35 & 70 & 20 & & 161 \\ \hline
**Alerting situations** & & & & & & \\ Touching head & 36 & & & & 948 & 984 \\ Touching back & 36 & & & & 948 & 984 \\ Touching torso & & 36 & & & 948 & 984 \\ Touching neck & & & & & 948 & 948 \\ Vomiting & & & & & 948 & 948 \\ Coughing/sneezing & 71 & & & & 948 & 1019 \\ Waving hands & 36 & & & & & 36 \\ Exercising & 36 & & & & 36 \\ Falling & 36 & & & & 948 & 984 \\ (Reject) & 36 & 35 & 170 & 120 & 6636 & 6997 \\ \hline
**Daily life actions** & & & & & & \\ Drinking & & & & 20 & 948 & 968 \\ Eating & & & & 20 & 948 & 968 \\ Reading & & & & 20 & 948 & 968 \\ Using phone & & & & 20 & 948 & 968 \\ Dressing/undressing & & 100 & & & 1896 & 1996 \\ Using laptop & & & & 20 & 948 & 968 \\ (Reject) & 287 & & & & 6636 & 6923 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Number of samples provided by each dataset that compose the merged dataset. are provided by two datasets only. Touching neck and vomiting are the only actions whose samples came from a single dataset (i.e., NTU [27]). From Table 4 we observe that the number of samples from the NTU dataset greatly outnumbers those from the other datasets. This could be a potential issue, as the resulting merged dataset will be strongly unbalanced, especially in some classes, toward samples from NTU. We tried to minimize the impact of this unbalancing on the merged dataset by adopting specific criteria for re-balancing the samples: the training set was subject to per-class balancing through sample duplication, with the objective of preventing a bias on given classes. The test set was balanced instead on the different datasets, with the objective of producing a heterogeneous and thus significant benchmark. More specifically, for the test set, we selected from each dataset 10 samples for each action. Notice that multiple actions given by one or more datasets can be used to compose a single action of our merged dataset. For the training set the applied rule was the previous one, except that we took the available samples (after excluding those used in test set) up to a maximum of 25.