## 3 Methodology
### Loss Functions  
Our total training loss consists of three components: (1) object label prediction loss, (2) hand pose estimation loss, and (3) action recognition loss.  
For the object label prediction task, HPEM predicts the object labels for each frame, and we perform supervised learning for this task by minimizing the cross-entropy loss function \(\mathrm{L_{obj}}\). \(\mathrm{L_{obj}}\) defines the difference between the actual object labeling distribution \(\mathrm{o^{gt}}=\left\{\mathrm{o^{gt}_{i}}\right\}_{i=1}^{\mathrm{n_{a}}}\) and the predicted object labeling probability distribution \(\mathrm{o^{pre}}=\left\{\mathrm{o^{pre}_{i}}\right\}_{i=1}^{\mathrm{n_{a}}}\)  
\[L_{obj}(t)=-\sum_{i=1}^{n_{a}}O_{i}^{gt}log\left(O_{i}^{pre}\right) \tag{5}\]  
For the hand pose estimation, we perform supervised learning with L1 loss, aiming to minimize the L1 loss during the training process. The loss function \(\mathrm{L_{hand}(I)}\) compares the difference between the predicted joint coordinates \(\left(\mathrm{H^{pre}},\mathrm{H^{pre}_{dep}}\right)\) and the true value \(\left(\mathrm{H^{gt}},\mathrm{H^{gt}_{dep}}\right)\). The loss function is articulated as follows:  
\[L_{hand}(t)=\frac{1}{J}\left(\left\|H^{pre}-H^{gt}\right\|_{1}+\alpha_{1}\left\| H^{pre}_{dep}-H^{gt}_{dep}\right\|_{1}\right) \tag{6}\]  
where J represents the total number of joints in the hand. \(\mathrm{H^{pre}}\) and \(\mathrm{H^{pre}_{dep}}\) are the coordinates and depth of the joints in the 2D image plane predicted by the model, respectively. \(\mathrm{H^{gt}}\) and \(\mathrm{H^{gt}_{dep}}\) are the actual 2D joint coordinates and depth, respectively. \(\alpha_{1}\) is a parameter used to balance the different weights between 2D coordinate losses and depth losses.  
For the action recognition task, ARM aggregates frame-wise feature representations to predict the video's action class, and supervised learning is applied to train this task by minimizing the cross-entropy loss function. \(\mathrm{L_{action}}\) defines the difference between the actual action label distribution \(\mathrm{a^{gt}}=\left\{\mathrm{a^{gt}_{i}}\right\}_{i=1}^{\mathrm{n_{a}}}\) and the predicted action label probability distribution \(\mathrm{a^{pre}}=\left\{\mathrm{a^{pre}_{i}}\right\}_{i=1}^{\mathrm{n_{a}}}\):  
\[L_{action}(v)=-\sum_{i=1}^{n_{a}}a_{i}^{gt}log\left(a_{i}^{pre}\right) \tag{7}\]  
The total loss function of the total training loss combines the above three losses, using the weighting parameters \(\alpha_{2}\) and \(\alpha_{3}\) to control the relative importance of each part of the optimization process. We define the total training loss \(\mathrm{L_{total}}\) as follows:  
\[L_{total}=L_{action}(v)+\frac{1}{T}\sum_{\varpi=\varpi g_{i}(v)}\sum_{t\in \overline{v}}\Bigl{(}\alpha_{2}L_{obj}(t)+\alpha_{3}L_{hand}(t)\Bigr{)} \tag{8}\]  
where \(\alpha_{2}\) and \(\alpha_{3}\) are parameters that balance the object label prediction loss term and the hand pose estimation loss term. By employing the combined loss function, the model can jointly optimize object label prediction and hand pose estimation, thereby ensure the accuracy of the final action recognition task and further enhancing the overall stability and performance of the network.