# Joint Learning of Temporal Models to Handle Imbalanced Data for Human Activity Recognition
## 4 Experimental Setup and Evaluation
### Results  
In this section, the experimental results of the experiments of the proposed joint learning of temporal models are presented and discussed. The joint temporal model is compared with LSTM, 1D CNN, and hybrid 1D CNN+LSTM. Table 4 shows the F-scores of the proposed joint learning of temporal models when compared with the individual and hybrid learners from Ordonez Home A and B datasets. Regarding the Kasteren datasets A, B, and C, the F-scores of the proposed joint learning of temporal models compared with the LSTM, 1D CNN, and hybrid model are shown in Tables 5 and 6. The results show that the joint learning of temporal models outperforms the individual temporal and hybrid learners by more than 4% in total from all the datasets. Figures 7 and 8 show the improvement of F1-score results of minority classes from all the five datasets by the proposed method when compared with the individual learners and the hybrid learner. \begin{table}
\begin{tabular}{c} **Joint** \\ **Learning** \\ \end{tabular} \\ \hline
**Breakfast** & 82.27 & 78.43 & 86.79 & 87.05 & 78.65 & 77.10 & 73.91 & 82.74 \\ \hline
**Grooming** & 62.06 & 46.66 & 57.14 & 70.96 & 62.99 & 59.67 & 53.63 & 64.08 \\ \hline
**Leaving** & 89.90 & 88.60 & 88.39 & 91.73 & 96.43 & 97.31 & 96.48 & 98.19 \\ \hline
**Lunch** & 95.50 & 95.45 & 94.57 & 98.31 & 86.45 & 84.47 & 79.76 & 88.13 \\ \hline
**Showering** & 75.86 & 80.00 & 64.00 & 82.35 & 75.00 & 80.00 & 51.81 & 82.71 \\ \hline
**Sleeping** & 97.23 & 97.13 & 99.66 & 99.47 & 99.49 & 99.26 & 99.62 \\ \hline
**Snack** & 66.66 & 66.66 & 66.66 & 73.32 & 70.37 & 68.96 & 62.11 & 73.19 \\ \hline
**Spare\_Time/TV** & 97.79 & 95.93 & 97.28 & 98.97 & 95.81 & 94.51 & 95.51 & 96.60 \\ \hline
**Toileting** & 72.21 & 69.23 & 69.84 & 75.23 & 18.51 & 0.07 & 18.46 & 31.18 \\ \hline
**Dinner** & - & - & - & - & 40.00 & 34.28 & 29.41 & 50.27 \\ \hline
**Total** & 82.16 & 79.79 & 80.20 & 86.39 & 72.36 & 69.59 & 66.03 & 76.51 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: F1-score results of Ordonez Smart home datasets. Figure 8: F1-score results of only minority classes. Figure 7: F1-score results of only minority classes for Kasteren Home C.  
_Appl. Sci._**2020**, _10_, 5293 \begin{table}
\begin{tabular}{c c c c c} \hline \hline  & \multicolumn{3}{c}{**(a)** Home A} \\ \hline
**Activities** & **LSTM** & **1D CNN** & \
\begin{tabular}{c} **1D CNN+** \\ **LSTM** \\ \end{tabular} &
\begin{tabular}{c} **Joint** \\ **Learning** \\ \end{tabular} \\ \hline Brush\_teeth & 51.09 & 22.03 & 50.00 & 56.32 \\ \hline Get\_drink & 40.00 & 22.20 & 40.01 & 47.11 \\ \hline Get\_Snack & 30.14 & 22.22 & 28.57 & 43.36 \\ \hline Go\_to\_bed & 88.20 & 88.18 & 87.96 & 89.96 \\ \hline Leave\_house & 99.53 & 99.75 & 99.45 & 99.88 \\ \hline Prepare\_breakfast & 78.00 & 72.00 & 75.00 & 79.19 \\ \hline Prepare\_Dinner & 88.88 & 94.01 & 96.55 & 96.73 \\ \hline Take\_shower & 85.24 & 79.45 & 80.00 & 86.31 \\ \hline Use\_toilet & 60.86 & 56.60 & 57.69 & 63.33 \\ \hline
**Total** & 69.10 & 61.82 & 68.07 & 73.54 \\ \hline \multicolumn{4}{c}{**(b)** Home B} \\ \hline
**Activities** & **LSTM** & **1D CNN** & \
\begin{tabular}{c} **1D CNN+** \\ **LSTM** \\ \end{tabular} &
\begin{tabular}{c} **Joint** \\ **learning** \\ \end{tabular} \\ \hline Brush\_teeth & 0.00 & 0.00 & 16.66 & 28.57 \\ \hline Eat\_brunch & 91.42 & 89.28 & 91.22 & 92.12 \\ \hline Eat\_dinner & 88.00 & 88.88 & 83.33 & 88.91 \\ \hline Get\_a\_drink & 0.00 & 0.00 & 00.00 & 40.00 \\ \hline Go\_to\_bed & 99.08 & 99.20 & 99.28 & 99.66 \\ \hline Leaving\_the\_house & 95.7 & 90.89 & 88.09 & 98.72 \\ \hline Prepare\_brunch & 84.65 & 78.57 & 82.75 & 87.80 \\ \hline Get\_dressed & 26.66 & 0.00 & 15.38 & 49.63 \\ \hline Prepare\_dinner & 96.36 & 96.96 & 90.90 & 97.11 \\ \hline Take\_shower & 83.63 & 74.50 & 80.00 & 84.93 \\ \hline Use\_toilet & 40.00 & 25.00 & 16.66 & 51.33 \\ \hline Wash\_dishes & 74.33 & 72.72 & 66.66 & 77.72 \\ \hline
**Total** & 65.40 & 59.66 & 61.74 & 74.70 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: F1-score results of Kasteren Smart home datasets.