# Automatic Measurement of Inclination Angle of Utility Poles Using 2D Image and 3D Point Cloud
## 3 Experiment and Analysis
### Model Evaluation and Result Analysis  
The Mask R-CNN model is evaluated by average precision (_AP_). _AP_ refers to the area surrounded by precision-recall (_P_-_R_) curves and coordinates under a certain intersection union ratio (_IoU_), which is used to represent the average correctness of detection under different recall rates. For example, _AP\({}_{50}\)_ represents the value of _AP_ when the _IoU_ threshold is 0.5. The recall rate (_R_), precision (_P_), and _AP_ are displayed as  
\[R=\frac{TP}{TP+FN} \tag{21}\]  
\[P=\frac{TP}{TP+FP} \tag{22}\]  
\[AP=\int_{0}^{1}Prdr \tag{23}\]  
where, _TP_ represents the part correctly predicted as the pole, _FN_ represents the part where the pole is incorrectly predicted as the background, and _FP_ represents the part where the background is incorrectly predicted as the pole. _P(r)_ refers to the _P_-_R_ curve.  
Figure 14: Loss curve of image segmentation training.  
Figure 15: Point cloud segmentation training process. (**a**) Loss curve; (**b**) Accuracy curve.  
Table 1 shows the experimental results. The accuracy of the original Mask R-CNN for pole skeleton segmentation is relatively low, and the \(AP_{75}\) is significantly lower when the \(IoU\) is at a high threshold of 0.75. Compared with the original Mask R-CNN, the segmentation accuracy after expanding bounding box has been significantly improved. Among them, \(P\) is increased by 30%, and \(AP_{75}\) is increased by 50%, which proves that the correction of the bounding box has a greater impact on the segmentation accuracy. Only adding the CBAM attention mechanism can improve the detection effect of the pole to a certain extent. The combination of bounding box modified and CBAM attention mechanism has the best effect with \(AP_{50}\) and \(AP_{75}\), improving by 1% and 52%, respectively. When the \(IoU\) is 0.5, \(P\) and \(R\) increase by 32% and 0.5%, respectively, which reduces the error rate of pole recognition and increases the accuracy of pole segmentation.  
The test effects of the original model and the improved model are intuitively shown in Figure 16.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Method** & **P** & **R** & \(\mathbf{AP_{50}}\) & \(\mathbf{AP_{75}}\) \\ \hline Mask R-CNN & 63.3\% & 99.25\% & 90.55\% & 6.58\% \\ Mask R-CNN + box expanded & 93.16\% & 99.5\% & 90.11\% & 57.05\% \\ Mask R-CNN + CBAM & 67.23\% & 95.3\% & 90.62\% & 8.03\% \\ Mask R-CNN + box expanded + CBAM & 95.03\% & 99.7\% & 91.13\% & 58.15\% \\ \hline \hline \end{tabular}
\end{table}
Table 1: Comparison of segmentation effect.  
Figure 16: Visual display of original and improved effects. (**a**) Original Mask R-CNN segmentation effect; (**b**) Improved Mask R-CNN segmentation effect.  
The point cloud segmentation effect is evaluated by accuracy (\(ACC\)) and \(IoU\). After training, the accuracy and intersection over union of PointNet segmentation are 92.4% and 83.7%, respectively. \(ACC\) and \(IoU\) can be obtained from the following equations.  
\[ACC=\frac{PC_{t}}{PC_{p}} \tag{24}\]  
\[IoU=\frac{PC_{t}}{PC_{p}+PC_{gt}-PC_{t}} \tag{25}\]  
where, \(PC_{t}\) is the number of point clouds correctly predicted as the poles, \(PC_{p}\) is the number of point clouds predicted as the poles, and \(PC_{gt}\) is the number of point clouds that actually contain pole.  
After the central axis fitting and angle calculation, the overall detection results are shown as Figure 17, where the detected angle is the inclination angle in the camera coordinate system.