## 3 Methods
### Pre-Training and Contrastive Loss  
A well-designed pretext task for pre-training aims to learn network weights that are universally applicable and useful to multiple downstream tasks. In terms of architecture, speed of inference is a major consideration in some tasks where the network used is a lightweight network. In contrast, the success of pre-training relies on networks with an excess of parameters. In terms of data, a sufficiently large amount of data is one of the keys to successful pre-training, so a dataset like NTU60 or NTU120 is needed. Finally, in terms of loss design, a contrast loss function needs to be designed according to the pretext task. In Figure 3, we conclude the pre-training framework we explore in this paper, and call the pre-training framework JointContrast. Specifically, given a skeletal sample \(\mathbf{S}\), we first generate two new samples \(\mathbf{S}_{1}\) and \(\mathbf{S}_{2}\) aligned in the same world coordinate system by common data augmentation, including random flips, rotations, translations, and scaling. We then feed the samples \(\mathbf{S}_{1}\) and \(\mathbf{S}_{2}\) to the shared backbone network to extract spatio-temporal feature representations \(\mathbf{F}_{1}\) and \(\mathbf{F}_{2}\). Finally, a contrast loss applied to these two spatio-temporal features is defined: we minimize the feature distance between corresponding joints and maximize the feature distance between non-corresponding joints.  
InfoNCE, proposed in [40], is used by most of the unsupervised representational learning methods applied to scene understanding. By treating contrast learning as a dictionary query process, InfoNCE views contrast learning as a classification problem of classifying positive sample pairs into the same class and negative sample pairs into different classes, with a concrete implementation using \(SoftMax\) loss. Our proposed contrast loss function based on corresponding joints is an improvement on the original InfoNCE.  
\[\mathcal{L}=-\sum_{(i,j)\in\mathcal{P}}log\frac{exp(\mathbf{f}_{1}^{l}\cdot \mathbf{F}_{2}^{j}/\gamma)}{\sum_{(\cdot,k)\in\mathcal{P}}exp(\mathbf{F}_{1}^ {i}\cdot\mathbf{F}_{2}^{k}/\gamma)} \tag{11}\]  
where \(\mathcal{P}\) is the set of all positive joint pairs. In this form, we only consider joints that have at least one corresponding joint and do not use additional non-corresponding joints as negative. For a positive joint pair \((i,j)\in\mathcal{P}\), the joint feature \(\mathbf{F}_{1}^{i}\) will be used as the query and \(\mathbf{F}_{2}^{i}\) will be used as the positive key. In addition, we use all joint features \(\mathbf{F}_{2}^{k}\) where \(\exists(\cdot,k)\in\mathcal{P}\) and \(k\
eq j\) as the set of negative keys. We summarize the above pre-training process as Algorithm 1.  
```
0: Backbone NN
0: Skeleton dataset \(X=\{\mathbf{x}_{i}\}\)
0: Channels for feature maps \(D\)
0: Pre-training parameters of the backbone network for Each skeletal sample in the dataset \(\mathbf{x}\)do  Generate two views from \(\mathbf{x}\), \(\mathbf{x}_{1}\) and \(\mathbf{x}_{2}\)  Sampling two transformations \(\mathbf{T}_{1},\mathbf{T}_{2}\)  Compute the features of the skeletal data \(\mathbf{f}_{1},\mathbf{f}_{2}\in\mathbb{R}^{N\times D}\) by \(\mathbf{f}_{1}=NN(\mathbb{T}_{1}(\mathbf{x}_{1}))\) and \(\mathbf{f}_{2}=NN(\mathbb{T}_{2}(\mathbf{x}_{2}))\)  Compute the loss and update the parameters of the network NN by back propagation using the contrast loss function \(\mathcal{L}(\mathbf{f}_{1},\mathbf{f}_{2})\) on the corresponding joints endfor
```  
**Algorithm 1** Joint contrast learning algorithm.