# Joint Learning of Temporal Models to Handle Imbalanced Data for Human Activity Recognition
## 4 Experimental Setup and Evaluation
### Results
When compared with LSTM, 1D CNN, and the hybrid model, the use of our proposed model increases the F-scores by 4%, 6%, and 6% in Ordonez home A, and by 4%, 6%, and 10% in Ordonez home B respectively. Regarding to the infrequent activities _Breakfast_, _Grooming_, _Lunch_, _shovering_, _toileting_, _snack_, _Dinner_, the proposed method improves F-scores by 4%, 8%, 3%, 7%, and 3% from Ordonez home A and by 4%, 1%, 2%, 2%, 11%, and 10% in Ordonez home B, respectively. This confirms the proposed model is capable of achieving better performance for the recognition of minority classes. When compared with LSTM, the proposed method improves F-scores by 4% from Ordonez home A, B, and Kasteren home A, as well as by 9% and 11% from Kasteren home B and C, respectively. When compared with 1D CNN, the proposed method improves F-scores by 5% from Ordonez home A, B, and by 11%, 15%, and 14% from Kasteren home A, B, and C, respectively. When compared with the hybrid model, the proposed method improves F-score by 6%, 10%, 5%, 12%, and 16% from Ordonez home A, B, and Kasteren home A, B, and C respectively. Regarding the F-score results from Kasteren datasets A, the results of the minority classes in addition to majority classes are increased using the proposed model when compared with the individual models and the hybrid model. The minority classes from home A are _Brush_teeth_, _Get_drink_, _Get_Snack_, _Prepare_Breakfast_, _Take_shower_, _Use_toilet_, those from home B are _Get_a_drink_, _Get_dressed_, _Use_toilet_, _Brush_teeth_, _Eat_dimer_, _Wash_dishes_, and those from home C are _Brush_teeth_, _Get_dressed_, _Get_snack_, _Get_drink_, _Go_to_bed_, _Prepare_Breakfast_, _Prepare_Dinner_, _Prepare_Lunch_, _Shave_, _Take_medication_, and _Use_toilet_. The experimental results show that the proposed joint learning of temporal models can improve the performance of minority classes besides the majority classes for human activity recognition. All of the models perform poorly for some minority activities, such as _Get_drink_ with only 20 samples, and _take medication_ with only six samples from Kasteren Home C. Firstly, a small number of samples are given high dimensional data with 315 features (21 sensors * 15 fuzzy temporal windows) from Kasteren Home C dataset, and one likely problem is the curse of dimensionality of the smart home datasets, since high dimension creates difficulties for the classifier to search. Secondly, the results indicate that there is not enough variation in the smart home data with respect to these two activities and the input features are non-informative and useless in the separation of these activities from the rest. To further improve these minority classes, imbalanced data could be handled from data level, \begin{table}
\begin{tabular}{l c c c c} \hline \hline
**Activities** & **LSTM** & **1D CNN** & **1D CNN+LSTM** & **Joint Learning** \\ \hline Eating & 74.28 & 80.00 & 80.00 & 82.70 \\ \hline Brush\_teeth & 47.61 & 58.33 & 50.00 & 62.50 \\ \hline Get\_dressed & 48.48 & 53.33 & 32.87 & 54.67 \\ \hline Get\_drink & 00.00 & 0.00 & 0.00 & 32.85 \\ \hline Get\_snack & 66.66 & 30.00 & 66.66 & 68.00 \\ \hline Go\_to\_bed & 93.65 & 94.68 & 91.48 & 94.86 \\ \hline Leave\_house & 92.86 & 91.81 & 91.64 & 98.96 \\ \hline Prepare\_Breakfast & 75.00 & 72.22 & 36.36 & 76.75 \\ \hline Prepare\_Dinner & 75.55 & 80.70 & 54.44 & 83.69 \\ \hline prepare\_Lunch & 70.00 & 72.72 & 72.72 & 74.35 \\ \hline Use\_Toilet\_Downstairs & 15.38 & 0.00 & 05.26 & 22.22 \\ \hline Use\_toilet\_upstairs & 13.33 & 0.00 & 16.66 & 18.38 \\ \hline Shave & 66.66 & 70.00 & 44.44 & 78.88 \\ \hline Take\_medication & 0.00 & 0.00 & 0.00 & 28.42 \\ \hline Take\_shower & 70.00 & 72.13 & 70.96 & 74.65 \\ \hline
**Total** & 53.96 & 51.68 & 47.56 & 64.46 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: F1-score results of Kasteren home C datasets.