# Detection of Parking Slots Based on Mask R-CNN
## 4 Method for Detecting Parking Slots Based on Mask R-CNN
### Build Mask R-CNN Training Model  
Mask R-CNN is a two-stage framework. In the first stage, the backbone network of Mask R-CNN (Resnet101 and FPN) extracts and combines the feature maps of the AVM image. The regional proposal network (RPN) is used to recommend anchors of marking-points, then anchors with high scores can be  
Figure 11: The annotation of the AVM image. Figure 12: Typical training samples. (**a,d**) Parallel and vertical parking slots, respectively; (**b,e**) segmentation of marking-points; (**c,f**) gray images with the mask of marking-points. found by filtering and correcting. In the second stage, the category and position of each anchor are predicted, and the corresponding mask of marking-points is generated. The overall network structure is shown in Figure 13.  
The above two stages are described in detail as follows: the powerful Resnet101 was selected to extract the feature of marking-point. We took 4 different scales of feature maps (\(C_{1}\),\(C_{2}\),\(C_{3}\),\(C_{4}\)) output by Residual Block. These feature maps were used to establish the feature pyramid of FPN network, and get new features (\(P_{2}\),\(P_{3}\),\(P_{4}\),\(P_{5}\),\(P_{6}\)), respectively. The feature combination process is shown in Equation (2). For \(i\) = 5, 4, 3, 2, \(U_{6}\) = 0:  
\[\left\{ \begin{array}{l}P_{i}^{\prime}=sum(U_{i+1},conv)\\ U_{i}=upsample(P_{i}^{\prime})\\ P_{i}=conv(P_{i}^{\prime})\\ P_{6}=pooling(P_{5})\\ \end{array} \right. \tag{2}\]  
where \(sum\) represents the element-by-element alignment operation, \(conv\) represents the convolution operation, _upsample_ represents the upsampling operation that doubles the length and width of the feature map and _pooling_ represents maximum pooling operation with a stride of 2. Anchors of marking-points are recommended by the region proposal network (RPN). A sliding window is used to slide on the five feature maps (\(P_{2}\),\(P_{3}\),\(P_{4}\),\(P_{5}\),\(P_{6}\)). After the sliding operation, the 2\(k\) dimension classification and 4\(k\) dimension position information are regressed to describe the correction values of \(k\) anchors. The correction values of each anchor include \(\Delta x\), \(\Delta y\), \(\Delta h\), \(\Delta W\), \(P\), where \(P\) is the confidence degree of foreground and background. Formula (3) is the correct formula for anchors:  
\[\left\{ \begin{array}{l}x^{\prime}=x+w\cdot\Delta x\\ y^{\prime}=y+h\cdot\Delta y\\ w^{\prime}=w\cdot\exp(\Delta w)\\ h^{\prime}=h\cdot\exp(\Delta h)\\ \end{array} \right. \tag{3}\]  
where (\(x\),\(y\)) is the coordinate of anchor center point, \(w\) and \(h\) are the width and height of anchor, respectively, and \(x^{\prime}\), \(y^{\prime}\), \(w^{\prime}\), \(h^{\prime}\) are corresponding correction values. The correction process is shown in Figure 14a. After the correction of a large number of anchors is completed, the non-maximum suppression method (NMS) [31] is used to retain anchors with high foreground scores and transfer them to the next stage, as shown in Figure 14b. Figure 13: Overall Mask R-CNN structure chart. To map the ROI in different sizes of anchors to the feature map of fixed size, ROI Align is proposed to select four regular positions in each ROI block, while double-line interpolation is used to calculate the exact value of each position and summarize the results. Finally, uniform ROIs are classified, the position of anchors is regressed and the mask of marking-point is generated by FCN. As shown in Figure 15a,b, Figure 16 shows the AVM image with marking-points detected by Mask R-CNN. Figure 16: AVM images with marking-points detected by Mask R-CNN. (**a**) Vertical parking slots; (**b**) parallel parking slots; (**c**) slanted slots. Figure 14: Anchors generation and filtering. (**a**) The dashed lines indicate the top 100 anchors with the highest foreground score recommended by RPN, the solid lines represent the corresponding anchors corrected by the regression information; (**b**) after non-maximum suppression method (NMS) filtering. Figure 15: Detection of marking-points.