# Insulator String Detection Method Based on the InST-Net Network

Authors: Zhuo Haoze; Han Jiaming; Zhou Guoxing; Yang Zhong
Journal: Mathematical Problems in Engineering
 (2022)

---

## Abstract

Aiming at the problem of detecting insulator strings in aerial images, a detection method of insulator strings based on the InST-Net network is proposed in this paper. First, the ResNet50 network pretrained on the ImageNet dataset is used as the backbone network for insulator string feature extraction. Subsequently, for insulator strings of different imaging sizes in the image, three detection branches are designed based on the design ideas of the existing YOLO model. Finally, an SPP module is adopted to improve the feature extraction capability of each detection branch of the proposed InST-Net network. The experimental results show that the InST-Net network detection accuracy rate reaches 90.63%, which is higher than that of the four classic one-stage target detection networks and the existing insulator string detection network.


---

## Body

## 1. Introduction

Long-term exposure to the insulator string in the field environment will cause defects such as self-explosion [1] and flashover [2], which will cause operation failures of transmission lines [3, 4]. For a long time, the maintenance work of insulator strings has always relied on traditional human patrol inspection methods. However, the geographical span of the transmission line is large and the topography is more complicated, which cause the traditional manpower inspection method to be inefficient. To solve this problem, Chinese power grid companies are gradually adopting unmanned aerial vehicles equipped with cameras to carry out insulator string inspection operations, distinguishing the insulator strings in aerial images by the naked eye, then analyzing and judging their operating status.Accurate detection of insulator strings in aerial images is a prerequisite for insulator defect analysis. With the rapid development of computer vision technologies, some researchers have begun to explore computer vision methods that can be used to automatically detect insulator strings in aerial images. By analyzing the imaging characteristics of insulator strings in aerial images, a series of computer vision-based insulator string detection methods are proposed to improve the efficiency of transmission line inspection. However, the background of aerial images is usually complex, and parts of the interferences often have similar characteristics with the insulator strings, which lead to the poor performances of the earlier methods [5–9]. In recent years, with the rapid development of deep learning theories, convolutional neural networks have been widely used in the field of target detection. Considering that convolutional neural networks have the advantages of high accuracy and robustness, some classic networks have played an important role in unmanned driving, military reconnaissance, and other engineering fields. Therefore, some researchers began to explore the application method of the convolutional neural network in insulator string detection.The work of [10, 11] proposes a method for detecting insulator strings based on the Faster-RCNN network [12]. However, the real-time performance of the Faster-RCNN network is usually poor that cannot meet the needs of real-time detection. In the work of [13], a Faster-RCNN network fused with SE (squeeze-and-excitation) module [14] is proposed to detect insulator strings in images. Experimental results show that the fusion channel attention mechanism can significantly improve the detection accuracy. However, the real-time performance of this method is still poor. To solve this problem, the work of [1] proposes a saliency-based insulator string detection method. This work deems that the saliency maps can effectively narrow the network search range, thereby improving the efficiency of insulator string detection. The experimental results on six hundred test images exhibit that compared with the traditional SVM and DPM models, the method proposed in this paper has the advantages of high accuracy and fast speed. However, this method still cannot meet the needs of real-time detection. The work of [15] believes that the imaging of some aerial images is relatively fuzzy, which is not conducive to the detection of insulator strings in the image. Therefore, the SRCNN network is adopted to reconstruct the fuzzy aerial images. Then, the YOLO network [16] is used to detect the insulator strings in images. The experimental results show that the detection performance of the YOLO network is better than that of the Faster-RCNN, SSD [17], and other networks.To solve the problems in the existing works, an insulator string detection method named InST-Net based on the previous work [18] is proposed in this paper. First, the ResNet network [20] pretrained on the ImageNet dataset [19] is used as the backbone for insulator string feature extraction. Then, inspired by the YOLO network, three detection branches are designed to detect the insulator strings of different sizes in the images. Finally, an SPP structure is used to improve the feature extraction ability of each detection branch. By using a large insulator string data set, the effectiveness of the InST-Net network is verified.


## 2. Algorithm Model

### 2.1. InST-Net Network

Previous insulator string detection mainly relied on the traditional feature description methods and the machine learning methods. These methods are greatly affected by the advantages and disadvantages of the preset feature extraction network. The extracted features often cannot accurately and comprehensively describe the imaging characteristics of the insulator string. In addition, the complex background of aerial images is an important factor affecting the detection accuracy of insulator strings. Various interference objects in the background often have relatively similar image characteristics with the insulator strings, which will have a greater impact on the robustness of existing detection methods. Different from the feature description method and machine learning method, the convolutional neural network extracts the image features in the ground-truth area of the insulator string and generates the high-level semantic features of the insulator string through the deepening of the network structure layer by layer, which can be more accurate and comprehensive, describing the image characteristics of the insulator string.Inspired by the abovementioned ideas, this paper proposes a method of insulator string detection based on the convolutional neural network. Related works show that the existing two-stage target detection network [12, 21, 22] has a poor real-time performance, which is not conducive to online detection of insulator strings. Although the one-stage target detection network [16, 23, 24] has better real-time performance, its network memory usage occupies a large amount of space, which is not conducive to the online deployment of various embedded systems. In order to take into account both of the detection accuracy, real-time performance, and memory usage, this paper is inspired by the design ideas of YOLOv3 and further proposes the InST-Net network on the basis of the literature [18]; the structure of InST-Net is shown in Table 1.Table 1
Structure of InST-Net.
L-DetM-DetS-DetInputImproved ResNet50InputConv + Upsample + RouteInputConv + Upsample + RouteNameSizeNameSizeNameSizeL1 Conv3 × 3 × 256M1 Conv3 × 3 × 128S1 Conv3 × 3 × 64L2 Conv1 × 1 × 512M2 Conv1 × 1 × 256S2 Conv1 × 1 × 128L3 Conv1 × 1 × 256M3 Conv1 × 1 × 128S3 Conv1 × 1 × 64SPPRoute + PoolingSPPRoute + PoolingSPPRoute + PoolingL4 Conv3 × 3 × 512M4 Conv3 × 3 × 256S4 Conv3 × 3 × 128L5 Conv1 × 1 × 256M5 Conv1 × 1 × 128S5 Conv1 × 1 × 64L6 Conv1 × 1 × 512M6 Conv1 × 1 × 256S6 Conv1 × 1 × 128L7 Conv3 × 3 × 256M7 Conv3 × 3 × 128S7 Conv3 × 3 × 64L8 Conv1 × 1 × 128M8 Conv1 × 1 × 64S8 Conv1 × 1 × 32L9 Conv1 × 1 × 24M9 Conv1 × 1 × 24S9 Conv1 × 1 × 24DetectionOutputDetectionOutputDetectionOutputFirst, due to the dual effects of aerial filming angle and aerial filming distance, the imaging form of the insulator string in the image is not fixed, so it is necessary to select a suitable feature extraction backbone network to extract the fine feature map of the insulator string. In this paper, by using transfer learning technology, the network model in the field of image classification is used as the feature extraction backbone network of the target detection network. Compared with the AlexNet [19], VGG [25], and GoogLeNet [26] networks, the ResNet series of networks take into account the four major advantages of deep network, i.e., depth, better feature semantics, easy learning and training, and small network memory usage. In the ResNet series of networks, the ResNet50 network has a smaller memory usage, faster calculation speed, and higher classification accuracy. Therefore, on the basis of adding a feature fixed layer at the end of the ResNet50 network, this paper selects this network as the feature extraction backbone network of InST-Net.Second, the size of the insulator string in the aerial image is not fixed, and all insulator strings can be roughly divided into three types: large size, middle size, and small size according to the imaging size. From the perspective of target detection network construction, it is necessary to construct different detection branches for insulator strings of different imaging sizes to improve the detection capability of the network. In response to this problem, this paper sets up multiscale detection branches for different imaging sizes in the InST-Net network and names them as L-Det, M-Det, and S-Det, respectively. To reduce the amount of network parameters, parts of the number of convolution kernel in each detection branch of the InST-Net network is halved on the basis of the network in [18].Finally, although the InST-Net network uses different detection branches for power components of different imaging sizes, from the perspective of network construction, more refined feature maps can be extracted for insulator strings of the same imaging size. Inspired by the works in [18, 27, 28], this paper embeds a spatial pyramid feature sampling structure (spatial pyramid pooling, SPP) for each detection branch of the InST-Net network, and the size of each layer of the pooling operation in this structure is set as 1 × 1, 5 × 5, 7 × 7, and 11 × 11. By fusing the composite features extracted by the pooling operation of each layer of the pyramid, composite features of each type of imaging size insulator string are generated, and the composite features of each detection branch can be used to improve the detection accuracy of the network.


### 2.2. Data Augmentation Scheme

Traditional data augmentation schemes generally use rotation, shift, and zoom operations for images in the training set, but these data augmentation methods still cannot simulate more complex image transformation states. To improve the complexity of the image to be trained and then improve the generalization ability of the InST-Net network, this paper adopts the mix-up operation [29] in the training process of the InST-Net network. The equation of the Mix-up is given as follows:(1)μx∗,y∗|xi,yi=1n∑jnEλδx∗=λxi+1−λxjy∗=λyi+1−λyj.Specifically, the parameterλ obeys the B distribution parameters, that is, λ∼Betaα,α, α∈0,+∞, and α is a hyperparameter. As α increases, the training error of the InST-Net network will increase, but at the same time, its generalization ability will also increase; when it tends to infinity, the InST-Net network training strategy tends to be the same as when mix-up is not set. xi and xj are the original input vector, yi and yj are the one-hot encoding labels, respectively. x∗ and y∗ satisfy the following relationship:(2)x∗=λxi+1−λxj,y∗=λyi+1−λyj.In the training process of the InST-Net network, for the insulator string “Input_image” contained in each batch participating in the training, it is fused with the randomly selected image “Random_Image” according to theλ proportion, and the fusion method is pixel-by-pixel addition to obtain participation in the “Train_Image”; the equation is given as follows:(3)Train_Image=λ⋅Input_Image+1−λ⋅Random_Image.When calculating the training loss function, the loss functions of “Input_Image” and “Random_Image” are calculated, respectively, and finally, the two loss functions are weighted and summed according to theλ proportion.


### 2.3. Training Optimization Strategy

To further improve the detection performance of the InST-Net network, the GIoU [30] training optimization strategy is used in the training process. The insulator string that needs to be detected in this work is taken as an example, the method’s steps are given as follows:(1)
Assume that the prediction box of the insulator string isBp=x1p,y1p,x2p,y2p, the ground truth of the insulator string is Bg=x1g,y1g,x2g,y2g and x2p>x1p, y2p>y1p;(2)
Calculate the minimum and maximum values of the insulator string prediction box on thex-axis and y-axis as follows:(4)x1np=minx1p,x2p,x2np=maxx1p,x2p,y1np=miny1p,y2p,y2np=maxy1p,y2p.(3)
Calculate the area enclosed by the insulator string prediction boxAp and Ground-truth Ag, respectively, as follows:(5)Ap=x2np−x1np⋅y2np−y1np,Ag=x2g−x1g⋅y2g−y1g.(4)
Calculate the areaI of the intersection between the insulator string prediction box and ground truth as follows:(6)I=x2I−x1I⋅y2I−y1I,ifx2I>x1I,y2I>y1I,0,otherwise.Specifically,x1I,y1I and x2I,y2I are calculated as follows:(7)x1I=maxx1np,x1g,x2I=minx2np,x2g,y1I=maxy1np,y1g,y2I=miny2np,y2g.(5)
Calculate the areaC that can include the prediction box of the insulator string and the minimum bounding rectangle of the ground truth as follows:(8)C=x2c−x1c⋅y2c−y1c.Specifically,x1C,y1C and x2C,y2C are calculated as follows:(9)x1C=minx1np,x2g,x2C=maxx2np,x2g,y1C=miny1np,y1g,y2C=maxy2np,y2g.(6)
The expression of GIoU is as follows:(10)GIoU=IoU−Ac−UAc.


## 2.1. InST-Net Network

Previous insulator string detection mainly relied on the traditional feature description methods and the machine learning methods. These methods are greatly affected by the advantages and disadvantages of the preset feature extraction network. The extracted features often cannot accurately and comprehensively describe the imaging characteristics of the insulator string. In addition, the complex background of aerial images is an important factor affecting the detection accuracy of insulator strings. Various interference objects in the background often have relatively similar image characteristics with the insulator strings, which will have a greater impact on the robustness of existing detection methods. Different from the feature description method and machine learning method, the convolutional neural network extracts the image features in the ground-truth area of the insulator string and generates the high-level semantic features of the insulator string through the deepening of the network structure layer by layer, which can be more accurate and comprehensive, describing the image characteristics of the insulator string.Inspired by the abovementioned ideas, this paper proposes a method of insulator string detection based on the convolutional neural network. Related works show that the existing two-stage target detection network [12, 21, 22] has a poor real-time performance, which is not conducive to online detection of insulator strings. Although the one-stage target detection network [16, 23, 24] has better real-time performance, its network memory usage occupies a large amount of space, which is not conducive to the online deployment of various embedded systems. In order to take into account both of the detection accuracy, real-time performance, and memory usage, this paper is inspired by the design ideas of YOLOv3 and further proposes the InST-Net network on the basis of the literature [18]; the structure of InST-Net is shown in Table 1.Table 1
Structure of InST-Net.
L-DetM-DetS-DetInputImproved ResNet50InputConv + Upsample + RouteInputConv + Upsample + RouteNameSizeNameSizeNameSizeL1 Conv3 × 3 × 256M1 Conv3 × 3 × 128S1 Conv3 × 3 × 64L2 Conv1 × 1 × 512M2 Conv1 × 1 × 256S2 Conv1 × 1 × 128L3 Conv1 × 1 × 256M3 Conv1 × 1 × 128S3 Conv1 × 1 × 64SPPRoute + PoolingSPPRoute + PoolingSPPRoute + PoolingL4 Conv3 × 3 × 512M4 Conv3 × 3 × 256S4 Conv3 × 3 × 128L5 Conv1 × 1 × 256M5 Conv1 × 1 × 128S5 Conv1 × 1 × 64L6 Conv1 × 1 × 512M6 Conv1 × 1 × 256S6 Conv1 × 1 × 128L7 Conv3 × 3 × 256M7 Conv3 × 3 × 128S7 Conv3 × 3 × 64L8 Conv1 × 1 × 128M8 Conv1 × 1 × 64S8 Conv1 × 1 × 32L9 Conv1 × 1 × 24M9 Conv1 × 1 × 24S9 Conv1 × 1 × 24DetectionOutputDetectionOutputDetectionOutputFirst, due to the dual effects of aerial filming angle and aerial filming distance, the imaging form of the insulator string in the image is not fixed, so it is necessary to select a suitable feature extraction backbone network to extract the fine feature map of the insulator string. In this paper, by using transfer learning technology, the network model in the field of image classification is used as the feature extraction backbone network of the target detection network. Compared with the AlexNet [19], VGG [25], and GoogLeNet [26] networks, the ResNet series of networks take into account the four major advantages of deep network, i.e., depth, better feature semantics, easy learning and training, and small network memory usage. In the ResNet series of networks, the ResNet50 network has a smaller memory usage, faster calculation speed, and higher classification accuracy. Therefore, on the basis of adding a feature fixed layer at the end of the ResNet50 network, this paper selects this network as the feature extraction backbone network of InST-Net.Second, the size of the insulator string in the aerial image is not fixed, and all insulator strings can be roughly divided into three types: large size, middle size, and small size according to the imaging size. From the perspective of target detection network construction, it is necessary to construct different detection branches for insulator strings of different imaging sizes to improve the detection capability of the network. In response to this problem, this paper sets up multiscale detection branches for different imaging sizes in the InST-Net network and names them as L-Det, M-Det, and S-Det, respectively. To reduce the amount of network parameters, parts of the number of convolution kernel in each detection branch of the InST-Net network is halved on the basis of the network in [18].Finally, although the InST-Net network uses different detection branches for power components of different imaging sizes, from the perspective of network construction, more refined feature maps can be extracted for insulator strings of the same imaging size. Inspired by the works in [18, 27, 28], this paper embeds a spatial pyramid feature sampling structure (spatial pyramid pooling, SPP) for each detection branch of the InST-Net network, and the size of each layer of the pooling operation in this structure is set as 1 × 1, 5 × 5, 7 × 7, and 11 × 11. By fusing the composite features extracted by the pooling operation of each layer of the pyramid, composite features of each type of imaging size insulator string are generated, and the composite features of each detection branch can be used to improve the detection accuracy of the network.


## 2.2. Data Augmentation Scheme

Traditional data augmentation schemes generally use rotation, shift, and zoom operations for images in the training set, but these data augmentation methods still cannot simulate more complex image transformation states. To improve the complexity of the image to be trained and then improve the generalization ability of the InST-Net network, this paper adopts the mix-up operation [29] in the training process of the InST-Net network. The equation of the Mix-up is given as follows:(1)μx∗,y∗|xi,yi=1n∑jnEλδx∗=λxi+1−λxjy∗=λyi+1−λyj.Specifically, the parameterλ obeys the B distribution parameters, that is, λ∼Betaα,α, α∈0,+∞, and α is a hyperparameter. As α increases, the training error of the InST-Net network will increase, but at the same time, its generalization ability will also increase; when it tends to infinity, the InST-Net network training strategy tends to be the same as when mix-up is not set. xi and xj are the original input vector, yi and yj are the one-hot encoding labels, respectively. x∗ and y∗ satisfy the following relationship:(2)x∗=λxi+1−λxj,y∗=λyi+1−λyj.In the training process of the InST-Net network, for the insulator string “Input_image” contained in each batch participating in the training, it is fused with the randomly selected image “Random_Image” according to theλ proportion, and the fusion method is pixel-by-pixel addition to obtain participation in the “Train_Image”; the equation is given as follows:(3)Train_Image=λ⋅Input_Image+1−λ⋅Random_Image.When calculating the training loss function, the loss functions of “Input_Image” and “Random_Image” are calculated, respectively, and finally, the two loss functions are weighted and summed according to theλ proportion.


## 2.3. Training Optimization Strategy

To further improve the detection performance of the InST-Net network, the GIoU [30] training optimization strategy is used in the training process. The insulator string that needs to be detected in this work is taken as an example, the method’s steps are given as follows:(1)
Assume that the prediction box of the insulator string isBp=x1p,y1p,x2p,y2p, the ground truth of the insulator string is Bg=x1g,y1g,x2g,y2g and x2p>x1p, y2p>y1p;(2)
Calculate the minimum and maximum values of the insulator string prediction box on thex-axis and y-axis as follows:(4)x1np=minx1p,x2p,x2np=maxx1p,x2p,y1np=miny1p,y2p,y2np=maxy1p,y2p.(3)
Calculate the area enclosed by the insulator string prediction boxAp and Ground-truth Ag, respectively, as follows:(5)Ap=x2np−x1np⋅y2np−y1np,Ag=x2g−x1g⋅y2g−y1g.(4)
Calculate the areaI of the intersection between the insulator string prediction box and ground truth as follows:(6)I=x2I−x1I⋅y2I−y1I,ifx2I>x1I,y2I>y1I,0,otherwise.Specifically,x1I,y1I and x2I,y2I are calculated as follows:(7)x1I=maxx1np,x1g,x2I=minx2np,x2g,y1I=maxy1np,y1g,y2I=miny2np,y2g.(5)
Calculate the areaC that can include the prediction box of the insulator string and the minimum bounding rectangle of the ground truth as follows:(8)C=x2c−x1c⋅y2c−y1c.Specifically,x1C,y1C and x2C,y2C are calculated as follows:(9)x1C=minx1np,x2g,x2C=maxx2np,x2g,y1C=miny1np,y1g,y2C=maxy2np,y2g.(6)
The expression of GIoU is as follows:(10)GIoU=IoU−Ac−UAc.


## 3. Experimental Results

This work uses Darknet [31] as the training and testing framework of the InST-Net network. To verify the accuracy of the InST-Net network, it is compared with the classic one-stage target detection networks: YOLOv3, YOLO9000, ResnetV2, YOLO-tiny, and an existing insulator string detection method [32]. The AP value commonly used in the field of target detection is selected as the comparison index. The AP value can be obtained by calculating the area under the precision-recall curve. The calculation methods of precision and recall are given as follows:(11)Precision=TPTP+FP,Recall=TPTP+FN.Specifically, TP indicates the true positive, FP represents the false positive, and FN is the false negative.The test results of the InST-Net network and five comparison networks are counted, and the results are shown in Figures1(a) and 1(b). Figure 1(a) shows that the AP value of the YOLOv3 network is 90.05%, the AP value of YOLO9000 is 89.83%, the AP value of ResnetV2 is 85.92%, the AP value of YOLO-tiny is 52.78%, and the AP value of the InST-Net network reaches 90.63%, respectively. Figure 1(b) shows that the AP values of the network proposed in [32] is 89.96%, which is lower than that of the InST-Net network. Therefore, it can be concluded that compared with the existing one-stage classic target detection network and an existing insulator string detection network, the InST-Net network proposed in this paper has the advantage of high detection accuracy.Figure 1
Comparison of AP value. (a) Comparison of InST-Net network and four classic one-stage detection networks and (b) comparison between the InST-Net network and the insulator string detection network proposed in [32].
(a)(b)For the UAV power inspection based on image processing, the ultimate goal is to realize the UAV online real-time detection of various electrical components in the aerial image. However, the memory capacity of airborne embedded devices is usually limited, and such devices often contain more than one detection algorithm or detection network. Therefore, when designing the insulator string detection network, it is necessary to reduce the network memory usage as much as possible while ensuring the detection accuracy, so as to alleviate the memory pressure of the embedded device. This paper further compares the memory usage of each comparison network, as shown in Figure2. It can be seen that although the AP values obtained by YOLOv3 and YOLO9000 are high, the memory usage of YOLOv3 is 235 MB, and the memory usage of YOLO9000 is 256 MB, which is the network with the highest memory usage of all the comparative networks. ResnetV2 and YOLO-tiny have a small memory usage, i.e., 87 MB and 33 MB, respectively, but the detection accuracy of ResnetV2 and YOLO-tiny networks is not as good as that of YOLOv3 and YOLO9000. The memory usage of the InST-Net network proposed in this paper is 141 MB, which is less than the memory usage of the insulator string detection network proposed in [32]. Combining the previous experimental results and analysis, it can be seen that the InST-Net network has the best balance between detection accuracy and memory usage.Figure 2
Memory usage comparison.On the basis of the abovementioned experiments, this paper further compares the real-time performance of the InST-Net and the five comparison networks, as shown in Figure3. By analyzing Figure 3, it can be seen that the average running time of the network proposed by YOLOv3 and [32] on an image is about 0.02 s; on the contrary, the average running time of YOLO9000, ResnetV2, YOLO-tiny, and InST-Net networks is about 0.01 s, the detection speed of the InST-Net is fast, and it has the potential of online detection of insulator strings.Figure 3
Average running time comparison.In order to further demonstrate the detection performance of the InST-Net network, three aerial images containing insulator strings are selected for visual exhibition, and the experimental results are shown in Figure4. Figure 4(a) shows an insulator string taken at a close range. The image features of the insulator string are obvious, and the InST-Net network can accurately detect this insulator string. Figure 4(b) shows a scene with poor imaging quality and an extreme filming angle. In this case, the detailed features of the insulator string are not clear and are limited by the filming angle. The characteristics of the insulator string in this image are quite different from those in Figure 4(a). The InST-Net network's ability to extract the high-level semantic features of the insulator string is beneficial. The insulator string in the image can still be accurately detected. Figure 4(c) shows an aerial image containing four insulator strings and the filming distance is far away. The image size of the insulator strings in this image is quite different. Based on the observations of the detection results, it can be seen that the InST-Net network can accurately detect the four insulator strings with different imaging sizes. In general, it can be concluded that the InST-Net network is robust and can accurately detect insulator strings in common aerial scenes.Figure 4
Visualization display. (a) Close-distance filming, (b) poor imaging quality and extreme angle, and (c) long-distance filming and different imaging sizes.
(a)(b)(c)


## 4. Conclusions

This paper proposes the InST-Net network for detecting insulator strings in aerial images. The InST-Net network can accurately detect insulator strings in the complex background of aerial images by extracting high-level semantic features of insulator strings. Experimental results show that the detection accuracy of the InST-Net network can reach 90.63%, which is higher than that of four classic one-stage target detection networks and an existing insulator string detection network. The average detection time of one image is only about 0.01 s, which has the potential for online detection of insulator strings. In addition, the memory usage of the InST-Net network is only about 141 MB, which can achieve a better balance between detection accuracy, real-time performance, and memory usage.



---

*Source: 4037131-2022-04-23.xml*