# Research on Fault Detection Algorithm of Electrical Equipment Based on Neural Network

Authors: Tianxiang Lei; Fangcheng Lv; Jiaomin Liu; Lei Zhang; Ti Zhou
Journal: Mathematical Problems in Engineering
 (2022)

---

## Abstract

With the rapid development of China’s electrical industry, the safe operation of electrical facilities is very important for social stability and people’s property safety. The failure detection method of conventional electrical equipment is hand detection, which has high experience of the detection person, lacks detection and error detection, and the detection efficiency is low. With the development of artificial intelligence technology, computer-assisted substation inspection is now possible, and substation inspection using an intelligent inspection robot equipped with an infrared device is one of the main substation inspection methods. In this paper, experiments are carried out using several neural network models. For example, if a faster region convolutional neural networks (RCNN) infrared detection model is employed, a good vg16 in the feature region of the extracted image takes into account the quality of the infrared image and the presence of multiple devices. Infrared images can be used to determine the basic features of various electronic devices. In order to detect targets in infrared images of electrical equipment, the fast RCNN target detection algorithm is used, and the overall recognition accuracy reaches 83.1%, and a good application effect is obtained.


---

## Body

## 1. Introduction

With the concept of “three types and two networks” put forward, the national power grid has continuously increased the consolidation of smart grid and the investment in grid construction and is committed to building a world-class energy network enterprise [1, 2]. Fault detection technology can find the faults of high-voltage equipment in time, which is very important for the safe operation of electrical equipment [3]. Among various fault detection technologies, infrared detection technology has good detection effects and the widest application range [4, 5]. This is usually detected through the use of various infrared detection devices to obtain the temperature value and temperature spatial distribution characteristics of high-voltage electrical devices and analyze the possibility and danger of high-voltage electrical devices [6]. This is an important means of detecting electrical equipment faults.The neural network is a new research direction in the field of artificial intelligence, which was introduced to make machine learning as close as possible to artificial intelligence [7]. Neural networks enable computers to learn the analysis of data samples autonomously [8]. A convolution neural network is established by imitating the visual perception mechanisms of organisms and has characteristics such as weight sharing or local connection [9].In this paper, we hope to use target detection, pattern recognition, and other means to analyze and process infrared images collected from cameras, infrared thermal imagers, intelligent roving robots, and other equipment, locate the positions of various electronic equipment in the images, and identify various faults in electrical equipment [10]. This is of great significance to ensuring the safe and stable operation of substation electrical equipment and has very important research value [11].


## 2. Convolution Neural Network

A convolution neural network is a recognition algorithm widely used in the field of computer vision in recent years and has a good application effect in the field of images [12]. Since the deep learning theory was put forward, the characteristic evaluation learning ability of convolution neural networks has attracted the attention of the majority of machine learning researchers. With the update and development of numerical computing equipment, various advanced deep neural network structures such as AlexNet, VGGNet, and GoogLeNet have been put forward [13, 14].In the task of image classification, the original infrared image is directly input by a convolution neural network, and feature learning and extraction are carried out by convolution layer operations, so there is no need to reduce the dimension of the original image. Figure1 shows the construction process of a convolution neural network.Figure 1
Construction process of a convolution neural network.


### 2.1. Convolution Layer

The convolution layer is the core part of the whole convolution neural network [15]. The shallow convolution layer can only extract low-level features such as edges, lines, and angles. The convolution network of each layer extracts more abstract features based on the information of the previous layer and finally obtains a volume feature map that is much smaller in spatial dimension than the original input image, but deeper in representation. If each neuron does not need to perceive the global image and senses the local pixels closely related to it, the convolution neural network will have an important pattern. The convolution neural network can obtain the global information by integrating the lower local information into the higher layer. As shown in Figure 2, the dot product + bias voltage of pixels is calculated by using formula (1) to slide the whole image by using the local correlation local receptive fields between layers through the convolution kernel. And a plurality of different convolution kernels can generate a plurality of feature graphs representing different features from the image through convolution calculation. The weight sharing in the convolution neural network greatly reduces the number of references in the network, which improves the efficiency of feature extraction of model images.Figure 2
Two-dimensional convolution process.The convolution calculation is shown in the following formula:(1)xjl=f∑i∈Njxil−1⊗kijl+bjl.


### 2.2. Activation Function

The activation function is used to do nonlinear processing on the extracted feature map. The function graph of the commonly used nonlinear activation function is shown in Figure3.Figure 3
Common activation functions.Sigmoid function is(2)sigmoidx=11+e−x.If the inputs are close to negative infinity and positive infinity, respectively, the sigmoid function of the green line in Figure3 has output values close to 0 and 1, respectively.Tanh function is(3)tanhx=ex−e−xex+e−x.ReLU function is(4)ReLUx=max0,x.


### 2.3. Pooling Layer

The pooling layer is the main method to reduce the dimension, and the pixel correlation theory of adjacent positions in the image is used. The original image input to the convolution neural network will not destroy the original structure after the operation, and the image information will hardly be lost. Through the choice of pixels in the window, the pooling mode can be divided into average pooling and maximum pooling. The calculation formulas of the two pooling methods refer to equations (5) and (6), respectively, and the implementation process is shown in Figure 4.(5)ymn=1S1S2∑j=0S2−1S1−1∑i=0xxm×S1+i,n×S2+j,(6)ymn=max0≤i<S1,0≤j<S2xm×S1+i,n×S2+j,where Y is the pooled output, M and N are the positions of the object pixels, S is the horizontal pooling step, S is the vertical pooling step, and X is the two-dimensional input matrix.Figure 4
Implementation of maximum pooling (left) and average pooling (right).


### 2.4. Full Connection Layer

All connection layers usually appear in the last 1-2 layers of a convolution neural network, and they are the most standard form of the neural network, which classifies image features extracted from the front-end network. As shown in Figure5, there is a weight relationship between any neuron in all connection layers and all neurons in the upper layer, and it is also helpful for each output node in the next layer to input each pixel value on the image in a linear arrangement.Figure 5
Schematic diagram of full connection layer.Dropout is introduced into all connection layers by randomly deleting some neurons in the hidden layer. Forward propagation calculation and inverse error correction can effectively prevent excessive appropriateness.


## 2.1. Convolution Layer

The convolution layer is the core part of the whole convolution neural network [15]. The shallow convolution layer can only extract low-level features such as edges, lines, and angles. The convolution network of each layer extracts more abstract features based on the information of the previous layer and finally obtains a volume feature map that is much smaller in spatial dimension than the original input image, but deeper in representation. If each neuron does not need to perceive the global image and senses the local pixels closely related to it, the convolution neural network will have an important pattern. The convolution neural network can obtain the global information by integrating the lower local information into the higher layer. As shown in Figure 2, the dot product + bias voltage of pixels is calculated by using formula (1) to slide the whole image by using the local correlation local receptive fields between layers through the convolution kernel. And a plurality of different convolution kernels can generate a plurality of feature graphs representing different features from the image through convolution calculation. The weight sharing in the convolution neural network greatly reduces the number of references in the network, which improves the efficiency of feature extraction of model images.Figure 2
Two-dimensional convolution process.The convolution calculation is shown in the following formula:(1)xjl=f∑i∈Njxil−1⊗kijl+bjl.


## 2.2. Activation Function

The activation function is used to do nonlinear processing on the extracted feature map. The function graph of the commonly used nonlinear activation function is shown in Figure3.Figure 3
Common activation functions.Sigmoid function is(2)sigmoidx=11+e−x.If the inputs are close to negative infinity and positive infinity, respectively, the sigmoid function of the green line in Figure3 has output values close to 0 and 1, respectively.Tanh function is(3)tanhx=ex−e−xex+e−x.ReLU function is(4)ReLUx=max0,x.


## 2.3. Pooling Layer

The pooling layer is the main method to reduce the dimension, and the pixel correlation theory of adjacent positions in the image is used. The original image input to the convolution neural network will not destroy the original structure after the operation, and the image information will hardly be lost. Through the choice of pixels in the window, the pooling mode can be divided into average pooling and maximum pooling. The calculation formulas of the two pooling methods refer to equations (5) and (6), respectively, and the implementation process is shown in Figure 4.(5)ymn=1S1S2∑j=0S2−1S1−1∑i=0xxm×S1+i,n×S2+j,(6)ymn=max0≤i<S1,0≤j<S2xm×S1+i,n×S2+j,where Y is the pooled output, M and N are the positions of the object pixels, S is the horizontal pooling step, S is the vertical pooling step, and X is the two-dimensional input matrix.Figure 4
Implementation of maximum pooling (left) and average pooling (right).


## 2.4. Full Connection Layer

All connection layers usually appear in the last 1-2 layers of a convolution neural network, and they are the most standard form of the neural network, which classifies image features extracted from the front-end network. As shown in Figure5, there is a weight relationship between any neuron in all connection layers and all neurons in the upper layer, and it is also helpful for each output node in the next layer to input each pixel value on the image in a linear arrangement.Figure 5
Schematic diagram of full connection layer.Dropout is introduced into all connection layers by randomly deleting some neurons in the hidden layer. Forward propagation calculation and inverse error correction can effectively prevent excessive appropriateness.


## 3. Infrared Thermal Imaging Detection of Electrical Equipment Based on Faster RCNN

### 3.1. Cascade Convolution Neural Network

In the two-stage algorithm, the detector needs to define positive and negative samples by using the crossover ratio, and IOU represents the crossover and parallel ratio between the predicted frame and the true frame. When the IOU value of the predicted frame and the true frame is higher than the set value, the algorithm determines that the predicted frame is a positive sample. The larger the IOU value, the higher the authenticity of the object determined by the prediction frame, and the greater the deviation of authenticity. Therefore, in the case where the obtained prediction frame is close to the truest frame, it is necessary to select the best threshold in the detection stage. The cascaded convolution neural network target detection algorithm proposed in 2018 solves the incorrectness of the detection frame and the overadaptation problem caused by the invalidation of samples in the detection process. The cascaded convolution neural network is composed of three detectors. After the input of each detector is restored to the previous detector box, the set value of the detector threshold is increased in turn, so that the increased value of the three thresholds is set to 0.50. The objective function is(7)fx,b=fT⋅fT−1⋅fT−2…f1x,b.


#### 3.1.1. Detection of Border Regression

As shown in Figure6 the regression box is generally represented by 4-dimensional vectors (X Y W H) and the red box in the figure represents the recommendation box selected via the location candidate network.Figure 6
Schematic diagram of border regression.Settingf (x, b) in equation (7) as the target image effect of the recommended box, and setting the four coordinate parameters of the remote box g as gx,gy,gw,gh, then the four distance vectors are defined for x as(8)δx=gx−bxbw,δy=gy−bybh,δw=loggwbw,δh=logghbh.


#### 3.1.2. Detection Classification

The function of the classifier is basically a posterior probability and it is expressed as(9)htx=py=t/x.Wheret represents different stages, y is the category label, and the classification uses a cross-entropy loss function.(10)Lcls=−yilogyi′+1−yilog1−yi′.Here,Yi in the formula is the true label probability that the i-th sample belongs to a certain category.The total loss of the cascaded convolution neural network algorithm is the common determination of border classification loss and detection box regression loss.(11)Lxt,g=Lclshtxt,yt+λyt≥1Llocftxt,bt,g.


### 3.2. Infrared Thermal Imaging Detection Model of Faster RCNN Electrical Equipment

Infrared images collected by infrared devices such as infrared cameras and intelligent roving robots contain many devices. In addition, it is affected by the environment, light, the photography angle, and other factors. The faster RCNN neural network model can extract the deep features adapted to the input image and realize the correct positioning and device type recognition of various electronic devices in infrared images. In this paper, in order to ensure the accuracy and effectiveness of model feature extraction, a selective search algorithm is used to support the determination and labeling of target types and positions in samples. In order to improve the learning effect of parameters, a multilabeled infrared image data set is constructed. This improves the engineering practicability of infrared thermal imaging detection of electronic equipment.In this paper, the infrared image of any size P timesQ is scaled to a fixed size M times N, then sent to the V6616 model for feature extraction, and a faster RCNN network structure diagram is used to generate the feature map.


#### 3.2.1. Feature Extraction Based on Transfer Learning

The traditional machine learning framework usually needs to meet two conditions: the number of training samples is sufficient, and the distribution of test samples and training samples is the same. With the development of artificial intelligence technology, the application field of machine learning is expanding. In some areas, the number of samples is limited, and the recognition accuracy is low after training and learning, which is prone to overadaptation. With the emergence and continuous development of transitional learning, the process of machine learning has changed from zero to learning mode and accumulated learning mode. Therefore, instead of using the initial parameter method, this paper adopts the mobile learning method to fine-tune the weighting value of the whole network on the infrared data set and update all the parameters in the model, as shown in Figure7 so as to adapt to the new classification task and better extract the features of infrared images.Figure 7
Classification process using a pretraining model.Figure8 shows the accuracy curves of CIFAR datasets for two VG16 models. When using the VG16 pretraining model to extract image features, the volume network of each layer extracts more abstract features based on the information of the previous layer.Figure 8
Performance curves of different training models.


#### 3.2.2. RPN Area Proposal Generation

According to the infrared image settings, 9 anchor boxes of different sizes and scales are generated (1 : 1, 1 : 2, 2 : 1) for each pixel in the feature map, as shown in Figure9.Figure 9
Anchor box.As shown in Figure10, red box A represents the original target location generated by the RPN network, and the green box g represents the real target location. Box regression explores the relationship between the regression windows G′ that map the original target A to obtain close to the real target G.Figure 10
Border regression.The simpler idea is the translation and scaling, and the calculation formula is as follows:(12)Gx′=Aw⋅dxA+Ax,Gy′=Ah⋅dyA+Ay,Gw′=Aw⋅expdwA,Gh′=Ah⋅expdhA.Looking at the above formula, we need to learn four transformations ofdx(A), dy(A), dW(A), and dh(A). If the difference between the target A of the input source and the actual target G is small, the transformation can fine-tune the target region by linear regression modeling. For a given input eigenvector X (X, Y, W, H), the set of learning parameters W makes the value after linear regression very close to the true value Y, that is, Y = WX.


#### 3.2.3. Classification and Border Correction

ROI pooling combines the feature maps obtained in VG16 to generate regional recommended feature maps and converts inputs with different sizes into outputs with fixed lengths to obtain recommended feature maps with fixed sizes. Then, all-region recommendations in the image are identified, and the category to which the output region recommendations belong is proposed.


### 3.3. Infrared Thermal Imaging Detection of Improved Faster RCNN Electrical Equipment

#### 3.3.1. Nonmaximum Suppression

Nonmaximum inhibition is an element of nonmaximum inhibition, which can be understood as local maximum exploration. This part represents the adjacent region, which has two variable parameters. One is the dimension of the adjacent region, and the other is the size of the adjacent region.Target detection generally uses window sliding to generate multiple region recommendations in infrared images and calculates the repetition (IoU) of other region recommendations and current region recommendations. The IoU’s expansion is intersection over union, and the crossover and parallel ratios between the prediction frame and the true frame are calculated as shown in the following equation:(13)IoU=areaC∩areaGareaC∪areaG.For the same electrical equipment, it is possible to recommend several high-scoring areas, but the one with the best effect is enough.Non-maximum suppression is actually an iterative process that deals with the cases of six regions proposedA, B, C, D, E, and F, from the order of small classification scores to the order of large classification scores.(1)
First, the proposalF of the maximum score area is selected, and it is judged whether the overlap rate IoU of A ∼ E and F is greater than a certain set threshold.(2)
If the overlap ofA, C, and F exceeds a threshold, A and C are deleted, after which F is marked as the recommendation of the region initially retained.(3)
E with the highest score is selected from the remaining rectangular frames B, D, and E, the overlap degree is judged between E and B and D, and marked it as the second area recommendation keeping E.(4)
The above steps are repeated to find the recommendations for all remaining areas.Generally, the connection layer or classifier after neural network accumulation needs a fixed input size and needs to crop or warp the input data, which leads to data loss or geometric distortion. Furthermore, the proportion of the apparatus in the infrared image varies according to different imaging distances, different angles, different sizes of the electrical apparatus, and the like. To solve this problem, faster RCNN processes images with the following two actions to adapt to the multiscale problem of device images.


#### 3.3.2. Category Amendments

Referring to the idea of NMS, this paper improves faster RCNN and adds the classification correction function according to the previous algorithm, so that the model can correct the wrong equipment position type. The classification correction steps are as follows:(1)
It is suggested to classify all areas of the infrared image into device wholeA and device part B according to naming rules.(2)
The area with the smallest area is selected fromA to recommend A, and the area (Bj) is recommended from B. As shown in the following equation, Ai and an area overlap ratio are calculated for the recommended region (Bj) in B:(14)AoR=areaBj∩areaAiareaBj.When AoR <0.8,Bj judges that it does not belong to Ai, does not process, and recommends Bj+1 to the hyperbolic cosine value of the next region.(3)
The area recommendation inA is circulated in turn before the whole area recommendation correction is completed, and before the whole area recommendation correction is completed, the recognition effect diagram before and after the category correction is shown in Figure 11.Figure 11
Recognition effect before and after category correction: (a) improved front isolating switch connector identification error; (b) correct identification of disconnector connector before improvement; (c) improved recognition error of the front arrester porcelain sleeve; and (d) improved front arrester porcelain sleeve is correctly identified.
(a)(b)(c)(d)


## 3.1. Cascade Convolution Neural Network

In the two-stage algorithm, the detector needs to define positive and negative samples by using the crossover ratio, and IOU represents the crossover and parallel ratio between the predicted frame and the true frame. When the IOU value of the predicted frame and the true frame is higher than the set value, the algorithm determines that the predicted frame is a positive sample. The larger the IOU value, the higher the authenticity of the object determined by the prediction frame, and the greater the deviation of authenticity. Therefore, in the case where the obtained prediction frame is close to the truest frame, it is necessary to select the best threshold in the detection stage. The cascaded convolution neural network target detection algorithm proposed in 2018 solves the incorrectness of the detection frame and the overadaptation problem caused by the invalidation of samples in the detection process. The cascaded convolution neural network is composed of three detectors. After the input of each detector is restored to the previous detector box, the set value of the detector threshold is increased in turn, so that the increased value of the three thresholds is set to 0.50. The objective function is(7)fx,b=fT⋅fT−1⋅fT−2…f1x,b.


### 3.1.1. Detection of Border Regression

As shown in Figure6 the regression box is generally represented by 4-dimensional vectors (X Y W H) and the red box in the figure represents the recommendation box selected via the location candidate network.Figure 6
Schematic diagram of border regression.Settingf (x, b) in equation (7) as the target image effect of the recommended box, and setting the four coordinate parameters of the remote box g as gx,gy,gw,gh, then the four distance vectors are defined for x as(8)δx=gx−bxbw,δy=gy−bybh,δw=loggwbw,δh=logghbh.


### 3.1.2. Detection Classification

The function of the classifier is basically a posterior probability and it is expressed as(9)htx=py=t/x.Wheret represents different stages, y is the category label, and the classification uses a cross-entropy loss function.(10)Lcls=−yilogyi′+1−yilog1−yi′.Here,Yi in the formula is the true label probability that the i-th sample belongs to a certain category.The total loss of the cascaded convolution neural network algorithm is the common determination of border classification loss and detection box regression loss.(11)Lxt,g=Lclshtxt,yt+λyt≥1Llocftxt,bt,g.


## 3.1.1. Detection of Border Regression

As shown in Figure6 the regression box is generally represented by 4-dimensional vectors (X Y W H) and the red box in the figure represents the recommendation box selected via the location candidate network.Figure 6
Schematic diagram of border regression.Settingf (x, b) in equation (7) as the target image effect of the recommended box, and setting the four coordinate parameters of the remote box g as gx,gy,gw,gh, then the four distance vectors are defined for x as(8)δx=gx−bxbw,δy=gy−bybh,δw=loggwbw,δh=logghbh.


## 3.1.2. Detection Classification

The function of the classifier is basically a posterior probability and it is expressed as(9)htx=py=t/x.Wheret represents different stages, y is the category label, and the classification uses a cross-entropy loss function.(10)Lcls=−yilogyi′+1−yilog1−yi′.Here,Yi in the formula is the true label probability that the i-th sample belongs to a certain category.The total loss of the cascaded convolution neural network algorithm is the common determination of border classification loss and detection box regression loss.(11)Lxt,g=Lclshtxt,yt+λyt≥1Llocftxt,bt,g.


## 3.2. Infrared Thermal Imaging Detection Model of Faster RCNN Electrical Equipment

Infrared images collected by infrared devices such as infrared cameras and intelligent roving robots contain many devices. In addition, it is affected by the environment, light, the photography angle, and other factors. The faster RCNN neural network model can extract the deep features adapted to the input image and realize the correct positioning and device type recognition of various electronic devices in infrared images. In this paper, in order to ensure the accuracy and effectiveness of model feature extraction, a selective search algorithm is used to support the determination and labeling of target types and positions in samples. In order to improve the learning effect of parameters, a multilabeled infrared image data set is constructed. This improves the engineering practicability of infrared thermal imaging detection of electronic equipment.In this paper, the infrared image of any size P timesQ is scaled to a fixed size M times N, then sent to the V6616 model for feature extraction, and a faster RCNN network structure diagram is used to generate the feature map.


### 3.2.1. Feature Extraction Based on Transfer Learning

The traditional machine learning framework usually needs to meet two conditions: the number of training samples is sufficient, and the distribution of test samples and training samples is the same. With the development of artificial intelligence technology, the application field of machine learning is expanding. In some areas, the number of samples is limited, and the recognition accuracy is low after training and learning, which is prone to overadaptation. With the emergence and continuous development of transitional learning, the process of machine learning has changed from zero to learning mode and accumulated learning mode. Therefore, instead of using the initial parameter method, this paper adopts the mobile learning method to fine-tune the weighting value of the whole network on the infrared data set and update all the parameters in the model, as shown in Figure7 so as to adapt to the new classification task and better extract the features of infrared images.Figure 7
Classification process using a pretraining model.Figure8 shows the accuracy curves of CIFAR datasets for two VG16 models. When using the VG16 pretraining model to extract image features, the volume network of each layer extracts more abstract features based on the information of the previous layer.Figure 8
Performance curves of different training models.


### 3.2.2. RPN Area Proposal Generation

According to the infrared image settings, 9 anchor boxes of different sizes and scales are generated (1 : 1, 1 : 2, 2 : 1) for each pixel in the feature map, as shown in Figure9.Figure 9
Anchor box.As shown in Figure10, red box A represents the original target location generated by the RPN network, and the green box g represents the real target location. Box regression explores the relationship between the regression windows G′ that map the original target A to obtain close to the real target G.Figure 10
Border regression.The simpler idea is the translation and scaling, and the calculation formula is as follows:(12)Gx′=Aw⋅dxA+Ax,Gy′=Ah⋅dyA+Ay,Gw′=Aw⋅expdwA,Gh′=Ah⋅expdhA.Looking at the above formula, we need to learn four transformations ofdx(A), dy(A), dW(A), and dh(A). If the difference between the target A of the input source and the actual target G is small, the transformation can fine-tune the target region by linear regression modeling. For a given input eigenvector X (X, Y, W, H), the set of learning parameters W makes the value after linear regression very close to the true value Y, that is, Y = WX.


### 3.2.3. Classification and Border Correction

ROI pooling combines the feature maps obtained in VG16 to generate regional recommended feature maps and converts inputs with different sizes into outputs with fixed lengths to obtain recommended feature maps with fixed sizes. Then, all-region recommendations in the image are identified, and the category to which the output region recommendations belong is proposed.


## 3.2.1. Feature Extraction Based on Transfer Learning

The traditional machine learning framework usually needs to meet two conditions: the number of training samples is sufficient, and the distribution of test samples and training samples is the same. With the development of artificial intelligence technology, the application field of machine learning is expanding. In some areas, the number of samples is limited, and the recognition accuracy is low after training and learning, which is prone to overadaptation. With the emergence and continuous development of transitional learning, the process of machine learning has changed from zero to learning mode and accumulated learning mode. Therefore, instead of using the initial parameter method, this paper adopts the mobile learning method to fine-tune the weighting value of the whole network on the infrared data set and update all the parameters in the model, as shown in Figure7 so as to adapt to the new classification task and better extract the features of infrared images.Figure 7
Classification process using a pretraining model.Figure8 shows the accuracy curves of CIFAR datasets for two VG16 models. When using the VG16 pretraining model to extract image features, the volume network of each layer extracts more abstract features based on the information of the previous layer.Figure 8
Performance curves of different training models.


## 3.2.2. RPN Area Proposal Generation

According to the infrared image settings, 9 anchor boxes of different sizes and scales are generated (1 : 1, 1 : 2, 2 : 1) for each pixel in the feature map, as shown in Figure9.Figure 9
Anchor box.As shown in Figure10, red box A represents the original target location generated by the RPN network, and the green box g represents the real target location. Box regression explores the relationship between the regression windows G′ that map the original target A to obtain close to the real target G.Figure 10
Border regression.The simpler idea is the translation and scaling, and the calculation formula is as follows:(12)Gx′=Aw⋅dxA+Ax,Gy′=Ah⋅dyA+Ay,Gw′=Aw⋅expdwA,Gh′=Ah⋅expdhA.Looking at the above formula, we need to learn four transformations ofdx(A), dy(A), dW(A), and dh(A). If the difference between the target A of the input source and the actual target G is small, the transformation can fine-tune the target region by linear regression modeling. For a given input eigenvector X (X, Y, W, H), the set of learning parameters W makes the value after linear regression very close to the true value Y, that is, Y = WX.


## 3.2.3. Classification and Border Correction

ROI pooling combines the feature maps obtained in VG16 to generate regional recommended feature maps and converts inputs with different sizes into outputs with fixed lengths to obtain recommended feature maps with fixed sizes. Then, all-region recommendations in the image are identified, and the category to which the output region recommendations belong is proposed.


## 3.3. Infrared Thermal Imaging Detection of Improved Faster RCNN Electrical Equipment

### 3.3.1. Nonmaximum Suppression

Nonmaximum inhibition is an element of nonmaximum inhibition, which can be understood as local maximum exploration. This part represents the adjacent region, which has two variable parameters. One is the dimension of the adjacent region, and the other is the size of the adjacent region.Target detection generally uses window sliding to generate multiple region recommendations in infrared images and calculates the repetition (IoU) of other region recommendations and current region recommendations. The IoU’s expansion is intersection over union, and the crossover and parallel ratios between the prediction frame and the true frame are calculated as shown in the following equation:(13)IoU=areaC∩areaGareaC∪areaG.For the same electrical equipment, it is possible to recommend several high-scoring areas, but the one with the best effect is enough.Non-maximum suppression is actually an iterative process that deals with the cases of six regions proposedA, B, C, D, E, and F, from the order of small classification scores to the order of large classification scores.(1)
First, the proposalF of the maximum score area is selected, and it is judged whether the overlap rate IoU of A ∼ E and F is greater than a certain set threshold.(2)
If the overlap ofA, C, and F exceeds a threshold, A and C are deleted, after which F is marked as the recommendation of the region initially retained.(3)
E with the highest score is selected from the remaining rectangular frames B, D, and E, the overlap degree is judged between E and B and D, and marked it as the second area recommendation keeping E.(4)
The above steps are repeated to find the recommendations for all remaining areas.Generally, the connection layer or classifier after neural network accumulation needs a fixed input size and needs to crop or warp the input data, which leads to data loss or geometric distortion. Furthermore, the proportion of the apparatus in the infrared image varies according to different imaging distances, different angles, different sizes of the electrical apparatus, and the like. To solve this problem, faster RCNN processes images with the following two actions to adapt to the multiscale problem of device images.


### 3.3.2. Category Amendments

Referring to the idea of NMS, this paper improves faster RCNN and adds the classification correction function according to the previous algorithm, so that the model can correct the wrong equipment position type. The classification correction steps are as follows:(1)
It is suggested to classify all areas of the infrared image into device wholeA and device part B according to naming rules.(2)
The area with the smallest area is selected fromA to recommend A, and the area (Bj) is recommended from B. As shown in the following equation, Ai and an area overlap ratio are calculated for the recommended region (Bj) in B:(14)AoR=areaBj∩areaAiareaBj.When AoR <0.8,Bj judges that it does not belong to Ai, does not process, and recommends Bj+1 to the hyperbolic cosine value of the next region.(3)
The area recommendation inA is circulated in turn before the whole area recommendation correction is completed, and before the whole area recommendation correction is completed, the recognition effect diagram before and after the category correction is shown in Figure 11.Figure 11
Recognition effect before and after category correction: (a) improved front isolating switch connector identification error; (b) correct identification of disconnector connector before improvement; (c) improved recognition error of the front arrester porcelain sleeve; and (d) improved front arrester porcelain sleeve is correctly identified.
(a)(b)(c)(d)


## 3.3.1. Nonmaximum Suppression

Nonmaximum inhibition is an element of nonmaximum inhibition, which can be understood as local maximum exploration. This part represents the adjacent region, which has two variable parameters. One is the dimension of the adjacent region, and the other is the size of the adjacent region.Target detection generally uses window sliding to generate multiple region recommendations in infrared images and calculates the repetition (IoU) of other region recommendations and current region recommendations. The IoU’s expansion is intersection over union, and the crossover and parallel ratios between the prediction frame and the true frame are calculated as shown in the following equation:(13)IoU=areaC∩areaGareaC∪areaG.For the same electrical equipment, it is possible to recommend several high-scoring areas, but the one with the best effect is enough.Non-maximum suppression is actually an iterative process that deals with the cases of six regions proposedA, B, C, D, E, and F, from the order of small classification scores to the order of large classification scores.(1)
First, the proposalF of the maximum score area is selected, and it is judged whether the overlap rate IoU of A ∼ E and F is greater than a certain set threshold.(2)
If the overlap ofA, C, and F exceeds a threshold, A and C are deleted, after which F is marked as the recommendation of the region initially retained.(3)
E with the highest score is selected from the remaining rectangular frames B, D, and E, the overlap degree is judged between E and B and D, and marked it as the second area recommendation keeping E.(4)
The above steps are repeated to find the recommendations for all remaining areas.Generally, the connection layer or classifier after neural network accumulation needs a fixed input size and needs to crop or warp the input data, which leads to data loss or geometric distortion. Furthermore, the proportion of the apparatus in the infrared image varies according to different imaging distances, different angles, different sizes of the electrical apparatus, and the like. To solve this problem, faster RCNN processes images with the following two actions to adapt to the multiscale problem of device images.


## 3.3.2. Category Amendments

Referring to the idea of NMS, this paper improves faster RCNN and adds the classification correction function according to the previous algorithm, so that the model can correct the wrong equipment position type. The classification correction steps are as follows:(1)
It is suggested to classify all areas of the infrared image into device wholeA and device part B according to naming rules.(2)
The area with the smallest area is selected fromA to recommend A, and the area (Bj) is recommended from B. As shown in the following equation, Ai and an area overlap ratio are calculated for the recommended region (Bj) in B:(14)AoR=areaBj∩areaAiareaBj.When AoR <0.8,Bj judges that it does not belong to Ai, does not process, and recommends Bj+1 to the hyperbolic cosine value of the next region.(3)
The area recommendation inA is circulated in turn before the whole area recommendation correction is completed, and before the whole area recommendation correction is completed, the recognition effect diagram before and after the category correction is shown in Figure 11.Figure 11
Recognition effect before and after category correction: (a) improved front isolating switch connector identification error; (b) correct identification of disconnector connector before improvement; (c) improved recognition error of the front arrester porcelain sleeve; and (d) improved front arrester porcelain sleeve is correctly identified.
(a)(b)(c)(d)


## 4. Experiment

### 4.1. Experimental Environment and Platform

The faster RCNN network model used in this paper uses LabelImg tools to display infrared data sets. The detailed software and hardware experimental environment is shown in Table1.Table 1
Experimental environment.
Software and hardware environmentNameOperating systemUbuntu 16.04 LTSDevelopment languagePython 2.7CPUIntel Xeon E3-1231 v3 @3.40GHZ quad-coreGPUNvidia GeForce GTX TITANX (12287 MB/Nvidia)Development toolsPyCharm, LabelImgDevelopment platformTensorFlow, OpenCVDeep learning is a new research direction in the field of machine learning, in which the mainstream source deep learning framework has been widely used by machine learning developers and achieved good application results.Based on TensorFlow deep learning framework, this paper constructs, trains and optimizes the faster RCNN network model. Compared with other in-depth learning frameworks, TensorFlow has the following main advantages:(1)
High availability. The TensorFlow workflow is relatively easy, the API is stable, compatibility is good, and TensorFlow and Numpy are perfectly combined, and many researchers who are proficient in Python data can easily obtain it. Unlike other libraries, TensorFlow does not require compile time, allowing users to reconsider more quickly. TensorFlow has established many advanced APIs such as Keras and SkFlow and using TensorFlow brings great benefits to users.(2)
High efficiency. Since the first release of TensorFlow, the efficiency of TensorFlow has improved with the efforts of more and more developers.(3)
Strong support. TensorFlow hopes for Google’s support, and Google has invested a lot of energy in the development of TensorFlow, which has become a common language for machine learning researchers and developers.


### 4.2. Infrared Data Set

At present, the common algorithm in fault diagnosis of electrical equipment is the research of certain electrical equipment (such as transformer, insulator, etc.), which is an undisclosed dataset. Particularly, the focus of this study is that datasets of infrared images of various electronic devices such as current inductors (CT), voltage inductors (PT), transformers, circuit breakers, and reactors are not disclosed.There are 74,377 visible and infrared image samples in the substation, which are divided into two voltage levels (500 kV and 220 kV) and eight kinds of equipment (transformer, circuit breaker, CT, PT, GIS, high resistance, reactance, and lightning arrester) by LabelImg tools. The infrared images collected from various models and positions are classified, sorted, and displayed, and finally, 27,588 infrared images with good quality can be screened, as shown in Table2.Table 2
Sample sources.
SubstationTotal sampleScreened sampleYidu station4311316219Tuan lam station3126411369Total7437727588


### 4.3. Comparison of Recognition Accuracy

In this paper, the recognition results of 5519 infrared test images of the test group are counted, and the recognition accuracy of voltage inductor, current inductor, and circuit breaker with a large number of samples is selected, as shown in Table3, which illustrates the application effect before and after the improvement of faster RCNN algorithm.Table 3
Comparison of recognition accuracy before and after improvement of faster RCNN algorithm.
Electrical equipmentFaster RCNN (%)Improved faster RCNN (%)Lightning rod84.795.4Voltage transformer74.593.6Current transformer79.390.1Circuit breaker81.391.2It can be seen from Table3 that through the improvement of the faster RCNN algorithm, the recognition accuracy of the four devices has been improved to different degrees due to the improvement of the faster RCNN algorithm, the increase in the classification correction function, and the significant improvement in the recognition accuracy of electrical equipment parts. The recognition accuracy of the whole electrical equipment of the faster RCNN algorithm has been improved to 92.8%, and the comparison results are shown in Figure 12.Figure 12
Comparison of overall recognition accuracy before and after faster RCNN improvement.


### 4.4. Feature Extraction Network Comparison

Experimentally, the performance of VG16, ResNet 50, and ResNet 101 feature extraction networks was compared. Table4 shows the performance comparison experimental results of the three feature extraction networks.Table 4
Feature extraction network performance comparison.
CNN modelInputAccuracy (%)Running time (h)VGG-16224∗22492.84.4ResNet50224∗22491.86.8ResNet101224∗22493.57.6The experimental results show that, compared with the VG16 method, the recognition accuracy of the ResNet 101 method is increased by 0.65%, but the execution time is increased by 70%. ResNet 101 is deeper and needs more calculations in network training, but it also needs higher calculation efficiency and more storage space. The depth of ResNet50 is also deeper than that of VG16. Theoretically, the accuracy should be higher than that of VG16, but the experimental results are the opposite. The experimental results show that the recognition performance of ResNet50 on infrared data sets is 1.2% lower than that of VG16.


## 4.1. Experimental Environment and Platform

The faster RCNN network model used in this paper uses LabelImg tools to display infrared data sets. The detailed software and hardware experimental environment is shown in Table1.Table 1
Experimental environment.
Software and hardware environmentNameOperating systemUbuntu 16.04 LTSDevelopment languagePython 2.7CPUIntel Xeon E3-1231 v3 @3.40GHZ quad-coreGPUNvidia GeForce GTX TITANX (12287 MB/Nvidia)Development toolsPyCharm, LabelImgDevelopment platformTensorFlow, OpenCVDeep learning is a new research direction in the field of machine learning, in which the mainstream source deep learning framework has been widely used by machine learning developers and achieved good application results.Based on TensorFlow deep learning framework, this paper constructs, trains and optimizes the faster RCNN network model. Compared with other in-depth learning frameworks, TensorFlow has the following main advantages:(1)
High availability. The TensorFlow workflow is relatively easy, the API is stable, compatibility is good, and TensorFlow and Numpy are perfectly combined, and many researchers who are proficient in Python data can easily obtain it. Unlike other libraries, TensorFlow does not require compile time, allowing users to reconsider more quickly. TensorFlow has established many advanced APIs such as Keras and SkFlow and using TensorFlow brings great benefits to users.(2)
High efficiency. Since the first release of TensorFlow, the efficiency of TensorFlow has improved with the efforts of more and more developers.(3)
Strong support. TensorFlow hopes for Google’s support, and Google has invested a lot of energy in the development of TensorFlow, which has become a common language for machine learning researchers and developers.


## 4.2. Infrared Data Set

At present, the common algorithm in fault diagnosis of electrical equipment is the research of certain electrical equipment (such as transformer, insulator, etc.), which is an undisclosed dataset. Particularly, the focus of this study is that datasets of infrared images of various electronic devices such as current inductors (CT), voltage inductors (PT), transformers, circuit breakers, and reactors are not disclosed.There are 74,377 visible and infrared image samples in the substation, which are divided into two voltage levels (500 kV and 220 kV) and eight kinds of equipment (transformer, circuit breaker, CT, PT, GIS, high resistance, reactance, and lightning arrester) by LabelImg tools. The infrared images collected from various models and positions are classified, sorted, and displayed, and finally, 27,588 infrared images with good quality can be screened, as shown in Table2.Table 2
Sample sources.
SubstationTotal sampleScreened sampleYidu station4311316219Tuan lam station3126411369Total7437727588


## 4.3. Comparison of Recognition Accuracy

In this paper, the recognition results of 5519 infrared test images of the test group are counted, and the recognition accuracy of voltage inductor, current inductor, and circuit breaker with a large number of samples is selected, as shown in Table3, which illustrates the application effect before and after the improvement of faster RCNN algorithm.Table 3
Comparison of recognition accuracy before and after improvement of faster RCNN algorithm.
Electrical equipmentFaster RCNN (%)Improved faster RCNN (%)Lightning rod84.795.4Voltage transformer74.593.6Current transformer79.390.1Circuit breaker81.391.2It can be seen from Table3 that through the improvement of the faster RCNN algorithm, the recognition accuracy of the four devices has been improved to different degrees due to the improvement of the faster RCNN algorithm, the increase in the classification correction function, and the significant improvement in the recognition accuracy of electrical equipment parts. The recognition accuracy of the whole electrical equipment of the faster RCNN algorithm has been improved to 92.8%, and the comparison results are shown in Figure 12.Figure 12
Comparison of overall recognition accuracy before and after faster RCNN improvement.


## 4.4. Feature Extraction Network Comparison

Experimentally, the performance of VG16, ResNet 50, and ResNet 101 feature extraction networks was compared. Table4 shows the performance comparison experimental results of the three feature extraction networks.Table 4
Feature extraction network performance comparison.
CNN modelInputAccuracy (%)Running time (h)VGG-16224∗22492.84.4ResNet50224∗22491.86.8ResNet101224∗22493.57.6The experimental results show that, compared with the VG16 method, the recognition accuracy of the ResNet 101 method is increased by 0.65%, but the execution time is increased by 70%. ResNet 101 is deeper and needs more calculations in network training, but it also needs higher calculation efficiency and more storage space. The depth of ResNet50 is also deeper than that of VG16. Theoretically, the accuracy should be higher than that of VG16, but the experimental results are the opposite. The experimental results show that the recognition performance of ResNet50 on infrared data sets is 1.2% lower than that of VG16.


## 5. Conclusion

In this paper, the faster RCNN target detection algorithm is applied to the infrared detection of electronic equipment, and the infrared thermal detection model of fast RCNN electronic equipment is constructed, which can detect and detect more electronic equipment in infrared images and achieve good application results. However, due to the similar appearance of electrical equipment parts, the recognition effect of RCNN fast electrical equipment parts is low in the infrared thermal imaging model. In order to solve this problem, the faster RCNN is improved based on the idea of nonmaximum deletion, and the classification correction function is added on the basis of the traditional algorithm so that the model can correct the wrong position type of equipment and improve the recognition accuracy of electrical equipment in infrared images. We also introduce the experimental environment and platform as well as the creation and configuration of infrared data sets, describe the model recognition accuracy before and after the improved algorithm, and compare the performance of VG16. ResNet50 and ResNet101 are function extraction networks.



---

*Source: 9015796-2022-02-23.xml*