# A Robust and Efficient Method for Effective Facial Keypoint Detection
## 4 Experimental Results
### Main Results
#### 4.2.5 The Result of Distillation and Pruning Loss Weight Ratio  
Regarding experiments investigating the effect of loss weights on distillation combined with pruning, Figure 5 presents the results. We studied the impacts of different distillation and pruning loss weights on model performance. By adjusting the weight ratio \(\lambda_{dp}\) of these two losses, we aimed to minimize model size and computational cost while maintaining accuracy.  
\[\lambda_{dp}\in\{0.2,0.5,1,2,5\} \tag{10}\]  
The experimental results show that, under the conditions of a weight ratio of 1 and a regression pruning rate of 50%, the best performance is achieved with an MSE of 0.0445 when the distillation and pruning loss weights are equal. This balance allows the model to achieve compression while maintaining accuracy. When the distillation weight is too small, the MSE is 0.0486, indicating that the student model cannot effectively learn from the teacher model, resulting in deteriorated performance. Conversely, when the distillation weight is  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Backbone** & **Algorithm** & **MSE** & \(t\)**-Test/\(p\)-Value** & **Levene-Test/\(p\)-Value** \\ \hline ResNet-50 & regression + heatmap & 0.0457 & T: 0.13, 0.48, 0.35 & L: 1.21, 1.55, 0.02 \\  & regression + heatmap & 0.0462 & P: 0.89, 0.62, 0.72 & P: 0.26, 0.21, 0.88 \\ MobileNet & regression + heatmap & 0.0462 & T: 0.64, 0.93, 0.29 & L: 0.03, 1.87, 1.28 \\  & regression + heatmap + & 0.0434 & P: 0.52, 0.34, 0.76 & P: 0.84, 0.34, 0.25 \\ MobileNet & \begin{tabular}{c} regression + heatmap + \\ distillation + heatmap + \\ distillation + pruning (50\%) \\ \end{tabular} & \begin{tabular}{c} 0.0434 \\ 0.0445 \\ 0.0445 \\ 0.0445 \\ 0.0445 \\ 0.0445 \\ 0.045 \\ \end{tabular} & \begin{tabular}{c} T: 0.49, 0.54, 0.05, 0.05 \\ 0.0445 \\ 0.0445 \\ 0.0445 \\ 0.045 \\ \end{tabular} & \begin{tabular}{c} L: 0.15, 0.02, 0.29 \\ P: 0.61, 0.58, 0.95 \\ 0.04, 0.05 \\ 0.045 \\ \end{tabular} \\ MobileNet & \begin{tabular}{c} regression + heatmap + \\ distillation + pruning (50\%) \\ \end{tabular} & \begin{tabular}{c} 0.0457 \\ 0.0457 \\ 0.0445 \\ 0.0445 \\ \end{tabular} & \begin{tabular}{c} T: 0.13, 0.48, 0.35 \\ P: 0.89, 0.62, 0.72 \\ P: 0.86, 0.21, 0.88 \\ \end{tabular} \\ MobileNet & \begin{tabular}{c} regression + heatmap + \\ distillation + pruning (50\%) \\ \end{tabular} & \begin{tabular}{c} 0.0462 \\ 0.0434 \\ 0.0434 \\ 0.0445 \\ \end{tabular} & \begin{tabular}{c} T: 0.49, 0.54, 0.05 \\ P: 0.61, 0.58, 0.95 \\ 0.045 \\ \end{tabular} & \begin{tabular}{c} L: 0.15, 0.02, 0.29 \\ P: 0.69, 0.88, 0.58 \\ \end{tabular} \\ MobileNet & \begin{tabular}{c} regression + heatmap + \\ distillation + pruning (50\%) \\ \end{tabular} & \
\begin{tabular}{c} T: 0.18, 0.63, 0.45 \\ P: 0.85, 0.52, 0.65 \\ \end{tabular} &
\begin{tabular}{c} L: 0.98, 0.42, 0.09 \\ P: 0.34, 0.51, 0.76 \\ \end{tabular} \\ \hline \hline \end{tabular}
\end{table}
Table 4: Performance comparison of distillation combined with pruning based on regression and heatmap.  
too large, the MSE is 0.0472, as the overemphasis on distillation loss neglects the role of pruning, failing to effectively reduce the model size and computational cost.