# Self-Attention Network for Human Pose Estimation
## 4 Experiment
### Experiments on 3D Pose of Human3.6M
#### 4.2.1 Ablation Study  
An ablation study was conducted using the Human3.6M test set and Protocol#1 was adopted. The self-attention network, extra 2D data, network depth and computation complexity were considered as shown in Tables 1 and 2.  
Self-Attention Network: Long-range dependency is important for articulated relations in human poses. Comparing to methods {_b,c_} and {_f,g_} in Table 1, performances increased by 4.3 mm MPJPE using ResNet-50 and 1 mm using ResNet-152 when considering SAN. SANs offer more long-term distance information. \(\delta=0.0\) at the beginning and increases in training. This means that long-term distance information is more important in higher level decision making processes. Considering method {_d,e_} in Table 1, the larger image resolution in our approach caused MPJPE to decrease by 1.2 mm. Figure 3 shows the results of the joint relationship over human joints with the change of \(\delta\). \(\delta=0.0\) means that the network has not introduced SAN, because the proximity information is easier to obtain at the beginning. Therefore, the number representing the feature gain between the joints is small. In comparison, the diagonal number is the largest, which means the current network is more dependent on neighboring features. With the increase in \(\delta\), the network introduced a more anatomical relationships and reweighted original feature maps. When \(\delta=0.8\), the larger number of means joints has strong correlation with other closed and symmetrical joints. The closed joints also have larger similarities than remote joints, such as joint 0, which has larger constraints with joint 7 than joint 15. Information transmits joints by joints such that the model will perceive more useful features. By adding this module, long-term distance information will be transformed between joints and compensate local content.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Method** & **Network** & **Input Size** & **2D Data** & **SAN** & **MPJPE** \\ \hline \(a\) & ResNet-50 & \(256\times 256\) & - & \(\times\) & 67.5 \\ \(b\) & ResNet-50 & \(256\times 256\) & MPII & \(\times\) & 62.2 \\ \(c\) & ResNet-50 & \(256\times 256\) & MPII & \(\surd\) & 57.9 \\ \(d\) & ResNet-50 & \(256\times 256\) & MPII COCO & \(\surd\) & 53.1 \\ \(e\) & ResNet-50 * & \(288\times 384\) & MPII COCO & \(\surd\) & 51.9 \\ \(f\) & ResNet-152 * & \(288\times 384\) & MPII COCO & \(\times\) & 49.6 \\ \(g\) & ResNet-152 * & \(288\times 384\) & MPII COCO & \(\surd\) & 48.6 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Ablative study on Human3.6M. Mean per joint position error (MPJPE) numbers are in mm.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Model** & **Params** & **Flops** & **MPJPE** \\ \hline ResNet-50 & 34M & 14.10G & 62.2 \\ ResNet-50+SAN & 39M & 14.43G & 57.9 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Ablative study on computation complexity of models with and without SAN.  
Joint learning framework: MPII and COCO datasets provide large-scale 2D key-point in the wild data. Comparing to methods {a,b} and {c,d} in Table 1, training with both 2D and 3D data provides significant performance gain--MPJPE dropped 5.3 mm when adding MPII dataset and 4.8 mm when adopting COCO dataset. This verifies the effectiveness of joint learning framework in our training process.  
Network depth: From method {e,f} in Table 1, the performance is enhanced by a deeper ResNet network. Changing network depth, MPJPE can drop by 3.3 mm from ResNet-50 to ResNet-152.  
Computation complexity: Table 2 compares the model with and without SAN in terms of parameter numbers and flops(Floating-point operations per second). The parameter of the original method is 34 M and our method is 39 M. The flops of the original method are 14.10 G and our method is 14.43 G. The parameter increases 5 M and flops increases 0.33 G when adding SAN, which leads to MPJPE dropping by 4.3 mm. This verifies that the SAN model with low computation complexity will achieve a better performance.