# Multi-Head TrajectoryCNN: A New Multi-Task Framework for Action Prediction
## 4 Experiments
### Comparison with State-of-the-Art  
Baselines. TrajectoryCNN and TrajectoryCNN-classification were selected as our baselines, among which \"TrajectoryCNN\" is for human motion prediction proposed in [8], and \"TrajectoryCNN-classification\" was modified based on TrajectoryCNN by replacing the convolutional layers in the decoder with two fully connected layers for predicting action semantics.  
Results. Table 1 shows the results compared with the baselines. The \"Accuracy\" column denotes the performance of action semantic prediction, and the \"Average error\" column denotes the average error across all time steps for human motion prediction. Compared with \"TrajectoryCNN\", our proposed method obtained the lowest prediction errors of human motion by significant margins of +10.12 and +11.07 for cross-view (CV) and cross-subject (CS) evaluations, respectively. Compared with \"TrajectoryCNN-classification\", the accuracy of our method increased by +6.81% and +3.6% for CV and CS evaluations, respectively. Our method achieved the best performance of human motion prediction and action semantic prediction via a unified model, showing the effectiveness of our multi-task framework. During backward propagation, the regression head and classification head are connected, and their tasks can be enhanced by each other. Therefore, our method thus obtains an improved performance of human motion prediction and action semantic predictions. The results also reveal these facts: (1) action semantic information can significantly help human motion prediction. (2) Human motion prediction also helps to predict action semantics. (3) The action semantics and future human motion can be integrated to improve the performance of each other.  
Figures 5 and 6 show more detailed results quantitatively and qualitatively. As shown in Figure 5, our proposed method obtained the lowest errors at all timestamps, especially at the later timestamps, again showing the effectiveness of our method. Similarly, as denoted in Figure 6, our method still achieved more reasonable predictions, especially in the later predictions. Our superior results in Figures 5 and 6 largely benefit from our multi-task framework that jointly models human motion and action semantics, which encourages the network to predict the human motion of the activities.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multicolumn{2}{c}{**CV**} & \multicolumn{2}{c}{**CS**} \\ \cline{2-5}  & **Accuracy** & **Average Error** & **Accuracy** & **Average Error** \\ \hline TrajectoryCNN & - & 80.97 & - & 91.78 \\ TrajectoryCNN- & 67.74\% & - & 60.22\% & - \\ \hline Multi-head & **74.55\% (+6.81\%)** & **70.85 (+10.12)** & **63.82\% (+3.6\%)** & **80.71 (+11.07)** \\ \hline \hline \end{tabular}
\end{table}
Table 1: Experimental results on NTU RGB+D, where “Average errors” denotes the average errors of MPJPE across all timestamps, “CV” denotes cross-view evaluation, and “CS” denotes cross-subject evaluation. The bold denotes the best results, and “+” denotes the improved performance of our method.