# An Edge-Guided Deep Learning Solar Panel Hotspot Thermal Image Segmentation Algorithm
## 2 Basic Framework of Mask R-CNN  
The main process of Mask R-CNN can be divided into four steps, and its structure is shown in Figure 1. The first step is to extract feature maps. Mask R-CNN uses Residual Neural Network (ResNet) ([PERSON] et al., 2014) as the backbone network to perform feature extraction at different scales on the input images after a series of convolution and pooling operations to obtain C1-1 to C1-5 layer features. The Feature Pyramid Network (FPN) is utilized to extract and fuse multi-scale features to obtain C3-2 to C3-6 feature maps of the input image at different levels. In the second step, the Region Proposal Network (RPN) is used to obtain the region proposal, where the effective feature layer sets candidate boxes of different proportions and sizes according to the size of different perceptual fields to obtain candidate regions that may contain targets. A Softmax classifier is used to determine whether the candidate targets belong to the foreground or the background. Afterwards, the candidate regions with a lower probability of hot spots and those with a higher overlap rate are filtered by non-maximum suppression (NMS). In the third step, ROI Align, a more accurate pooling method, is used to transform all the regions corresponding to the boxes generated and filtered by the RPN into fixed-size feature maps that align the candidate boxes. In the fourth step, the feature maps of the candidate regions are used for classification and segmentation. These feature maps are passed through two branches: one branch is used for classification and box regression through a series of fully connected layers, and the other branch is used to produce a mask that is consistent with the size and shape of the hot spot to segment the hot spot image by a segmentation mask-generating network consisting of a Full Convolution Network (FCN).  
Among them, focusing on the feature extraction stage, we use ResNet50 combined with FPN as the backbone network for feature extraction of PV hot spot images. Among them, ResNet50 is a convolutional neural network with a total of 50 convolutions, which can sequentially extract the low-level features (edges, etc.) and high-level features (hot spots, PV panels, etc.) to form five layers of feature maps with different sizes and dimensions. If only the feature map of the last layer of the residual network is used as the output of the feature extraction network, it is easy to ignore relatively small hot spots. Therefore, the FPN network is used to fuse the feature map from the bottom to the top layer so as to fully utilize the features extracted by ResNet50 at each stage. Specifically, each layer from C1-1 to C1-5 is convolved by \(1\times 1\) to obtain the same number of channels as the previous layer. The upper feature layer obtains feature layers with the same length and width as the feature layer of the next layer by up-sampling. Two feature layers are added together to obtain a fused new feature layer. After fusion, a 3 \(\times\) 3 convolution is used for processing, with the aim of eliminating the aliasing effect of up-sampling to obtain C3-2 to C3-5 feature layers. The C3-6 feature layer is obtained by maximizing the pooling of the C3-5 feature layer.  
Although the traditional Mask R-CNN network has powerful functions and can effectively realize hot spot detection and segmentation, there are still some problems when applied to thermal infrared images. For example, thermal infrared images have problems such as low resolution and unclear hot spot edges, which require more edge features, but it is difficult to enhance the edge features. There is a significant difference in the size of hot spots, which cannot adaptively extract regions of interest, resulting in the loss of the target. This paper focuses on researching these problems.