# Three-Dimensional Human Pose Estimation with Spatial-Temporal Interaction Enhancement Transformer
## 3 Method
### Cross-Domain Interaction
#### 3.3.1 Spatial Reconstruction  
In the first stage, we extracted features from the temporal and spatial domains, respectively. However, both types of information are extracted independently, and there is no interaction between them. To employ temporal action information to reconstruct the joint structural relationship, we need to inject the temporal features into the spatial domain. Before that, we must realize that the temporal and spatial features are inconsistent. In the temporal domain, each frame of a 2D pose sequence is represented by a token whose dimension is \(1\times D\), while in the spatial domain, the feature dimension is \(J\times C\), and each token represents one joint. This results in the information from different domains not being able to be directly interacted. To achieve cross-domain temporal information injection, we propose the SR block (see Figure 4a), which consists of the feature transforming unit (FTU), multi-head cross-attention (MCA), MSA, and MLP.  
We first utilized the FTU to convert the temporal features to the spatial domain. Specifically, a linear layer converts the number of temporal feature channels to \(J\cdot C\) and then passes through a LayerNorm layer. Finally, the channels are grouped and reshaped into \(J\times C\). After the above operations, the temporal features are successfully transformed from the temporal domain to the spatial domain (\(\widehat{Z_{l}^{T}}\rightarrow\{\widehat{Z_{l}^{T\to 51}},\widehat{Z_{l}^{T \to 52}},\cdots,\widehat{Z_{l}^{T\to 5N}}\}\)). Next, \(\widehat{Z_{l}^{T\to Si}}\) and spatial feature \(\widehat{Z_{l}^{Si}}\) are fed to the MCA. The MCA interacts the information  
Figure 4: Cross-Domain Interaction. (**a**) The Spatial Reconstruction (SR) block injects the temporal features into the spatial domain after transformation to adjust the spatial structure. (**b**) The Temporal Refinement (TR) block receives the reconstructed spatial features to further refine the temporal features.  
from different domains and has a similar structure to the MSA (see Figure 2c). The feature \(\widetilde{Z_{l}^{Si}}\) serves as the query, whereas the feature \(\widetilde{Z_{l}^{T\to Si}}\) serves as the key and value. We used MCA to inject temporal feature \(\widetilde{Z_{l}^{T\to Si}}\) into the spatial feature, then the result feature \(\widetilde{Z_{l}^{Si^{\prime}}}\) goes through the MSA and MLP to restructure the relationship between the joints, which can be formulated as Equation (6).  
\[\widetilde{Z_{l}^{Si^{\prime}}}=\widetilde{Z_{l-1}^{Si}}+MCA(LN( \widetilde{Z_{l-1}^{Si}}),LN(\widetilde{Z_{l-1}^{T\to Si}}),LN( \widetilde{Z_{l-1}^{T\to Si}})), \tag{6}\] \[\widetilde{Z_{l}^{Si^{\prime\prime}}}=\widetilde{Z_{l}^{Si^{\prime \prime}}}+MSA(LN(\widetilde{Z_{l}^{Si^{\prime}}})),\] \[\widetilde{Z_{l}^{Si}}=\widetilde{Z_{l}^{Si^{\prime\prime}}}+ MLP(LN(Z_{l}^{Si^{\prime\prime}})),\]  
where \(l\in[1,\cdots,L_{2}]\) is the index of CDI layers. After these operations, we successfully injected the temporal features into the spatial domain and obtained the final spatial reconstruction features \(\widetilde{Z_{L_{2}}^{Si}}\) from the SR block following the \(L_{2}\)-layer CDI module.