# Intelligent Diagnosis of Concrete Defects Based on Improved Mask R-CNN
## 2 Methods  
### Mask R-CNN  
Mask R-CNN constructs a mask branch based on Faster R-CNN, so it has both target detection functions and semantic segmentation functions and has become the pioneer of instance segmentation. Mask R-CNN adopts ResNet101 as the backbone for image feature extraction, which solved the problem that deep neural networks were difficult to train due to the exploding and vanishing gradient problem. The Mask R-CNN architecture is shown in Figure 1; it contains five parts:  
1. [label=(0)]
2. Feature extraction network: combined ResNet101 and FPNs to extract the multiscale feature map. As shown in Figure 2, the left is ResNet101, and the original image is convolved and pooled to obtain the feature figures C1-C5 of the five stages. Then, C5 is copied into P5, P5 is up-sampled, C4 is conducted dimensionality reduction and then C4 and P5 are fused to obtain P4, and so on, forming a top-down feature pyramid. Generally, convolutional neural networks (CNNs) directly use the feature maps of the last layer to predict the targets; although the feature map of the last layer has strong semantics, the resolution is relatively low, so the relatively small targetscannot be detected easily. FPN fuses the high semantic feature information of the higher layer with the high-resolution feature information of the lower layer, and can make the prediction on each feature layer, so that the features of smaller targets can be extracted more easily. 2. Region proposals: input feature map into region proposal network (RPN), take each pixel in the feature map as the center, map out nine different anchor boxes (which are formed by a free combination of three different aspect ratios (0.5, 1, 2) and three different pixel scales (1282, 2562 and 5122)) on the original image, to obtain multiple candidate regions of interest (ROIs); These candidate ROIs are judged by softmax on whether they contain targets, and bounding box regression is used to correct the position of anchors. The non-maximum suppression algorithm is used to filter out some candidate ROIs. Then, obtain region proposals. 3. ROI Align: through ROI Align, the region proposals are aligned to each pixel of the feature map, and each pixel of the feature map is aligned to fixed features. 4. Fully connected network: the region proposals which are aligned by pixel and fixed features are used for target classification and prediction box regression (target location). 5. Mask branch: generates a prediction box and classifies and masks the pixel points inside the box to obtain the semantic segmentation results. The loss function L of Mask R-CNN is shown in Equation (1). The loss function L contains three components: classification loss \(L_{cls}\), detection loss \(L_{box}\) and segmentation loss \(L_{mask}\). \[L=L_{cls}+L_{box}+L_{mask}=\frac{1}{N_{cls}}{\sum_{i}}l_{cls}({p_{i}},{p_{i}} ^{*})+\lambda\frac{1}{N_{r\!g}}{\sum_{i}}{p_{i}}^{*}L_{r\!g}(t_{i},{t_{i}}^{*} )+L_{mask} \tag{1}\]  
where,  
\[l_{cls}({p_{i}},{p_{i}}^{*})=-\lg[{p_{i}}^{*}{p_{i}}+(1-{p_{i}}^{*})(1-{p_{i}} )];L_{r\!g}=\frac{1}{N_{r\!g}}{\sum_{i}}{p_{i}}^{*}l_{r\!g}(t_{i},{t_{i}}^{*})\]  
\[l_{r\!g}(t_{i},{t_{i}}^{*})=Smooth_{\!L1}(x)=\left\{ \begin{matrix}0.5x^{2}, \,|x|<1\\ |x|-0.5,\,|x|\geq 1\end{matrix} ^{*}x=t_{i}-{t_{i}}^{*}\right.\]  
\[L_{mask}=-\sum_{i}\{t_{i}\lg[{\it sigmoid}(x_{i})]+(1-t_{i})\lg[1-{\it sigmoid }(x_{i})]\}\]  
Figure 1: Mask R-CNN model structure. where \(i\) is the index of the anchor box in each batch; \(p_{i}\) represents the predicted probability when the anchor point \(i\) is a certain anchor box. If the anchor box is positive, the true label \(p_{i}^{*}\) is 1; this means that the anchor box has a high overlap with a ground-truth object and the anchor box can be used to predict that object.