# RotJoint-Based Action Analyzer: A Robust Pose Comparison Pipeline
## 4 RotJoint-Based Pipeline
### What Are RotJoints?  
RotJoints, which represent the characterization of three rotational parameters for the 21 joints of the human body, are designed to circumvent the ambiguities, and allow for the parameters to be effectively visualized and interpreted meanwhile. The 21 joints include the head, neck, collarbone, shoulders, elbows, wrists, hands, spine, pelvis, hips, knees, ankles, and feet, with each joint's rotation characterized by pitch, yaw, and roll. The fundamental concept of Euler angles involves decomposing angular displacement into a sequence of three rotations around three mutually perpendicular axes. This decomposition makes Euler angles more intuitive and easier to interpret for understanding human motion. In the human pose estimation stage, the relative rotations of joint points can be obtained by sequentially rotating around the three axes (roll, pitch, yaw). We adopted the rotation order commonly used in the field of human pose estimation: roll \(\rightarrow\) pitch \(\rightarrow\) yaw. This sequence places the axis least likely to rotate by 90 degrees (pitch) in the middle, thereby minimizing the occurrence of gimbal lock issues and reducing the introduction of additional ambiguities.  
The distinction between our approach and traditional skeletal vector-based methods for action representation lies in our use of joint rotations in 3D space to define human posture and motion. Traditional methods, which use skeletal representations for similarity matching, are limited by inherent ambiguities. Our approach, on the other hand, is designed to circumvent these ambiguities, allowing for the parameters to be effectively visualized and interpreted.  
Given a human image \(I\in\mathbb{R}^{H\times W\times 3}\), we adopt a framework to perform full-body mesh recovery, estimating the corresponding body parameters \(\hat{Q}_{\text{body}}\). These parameters are then fed into an SMPL layer to obtain the final 3D whole-body human mesh. In detail, \(\hat{Q}_{\text{body}}\) contains the 3D body joint rotations \(\theta_{\text{body}}\in\mathbb{R}^{21\times 3}\).  
In other works, various networks have been designed to generate pose and body shape parameters as inputs for the SMPL model. Here, we adopt the architecture of the OSX model, simplifying it by removing the shape and unnecessary facial parameter regression parts. To generate the \(Q\) parameters essential for our RotJoint-based pose comparison, we process human images by dividing the input image \(I\) into fixed-size patches of \(M\), creating patches \(Q\in\mathbb{R}^{\frac{HW}{M^{2}}\times(M^{2}\times 3)}\). These patches undergo linear projection and are combined with position embeddings \(Q_{\epsilon}\in\mathbb{R}^{\frac{HW}{M^{2}}\times\mathbb{C}}\) to form feature tokens \(T_{f}\in\mathbb{R}^{\frac{HW}{M^{2}}\times\mathbb{C}}\). To incorporate body prior information, we concatenate these feature tokens with learnable body tokens \(T_{b}\in\mathbb{R}^{B\times\mathbb{C}}\). The concatenated tokens are then processed by a Transformer encoder, yielding updated tokens \(T^{\prime}_{b}\in\mathbb{R}^{B\times\mathbb{C}}\). The body parameters \(\theta_{\text{body}}\) are finally regressed from \(T^{\prime}_{b}\) using fully connected layers.  
In 2D pose estimation, viewpoint changes distort action recognition, while differences in body shapes and torsional movements can lead to misinterpretations. RotJoints, derived from a 3D mesh-based evaluation model, capture joint rotations relative to their parent nodes and joint axes, making the representation invariant to viewpoint changes and body shape variations. Additionally, by explicitly modeling rotational degrees of freedom around joint axes, RotJoints effectively capture torsional movements, improving the robustness and accuracy of pose comparison across diverse scenarios.