# MU R-CNN: A Two-Dimensional Code Instance Segmentation Network Based on Deep Learning
## 3 Our Contributions  
### Problem Presentation and Our Solution  
As mentioned above, in Industry 4.0 applications, QR tags are expected to have high positioning accuracy and fine texture semantic segmentation. According to the current research status, the existing instance segmentation algorithms based on deep learning has greater advantages in robustness and can adapt to a variety of complex application scenarios. However, instance segmentation is a challenging task, because it requires classification and location at the instance level, while semantic segmentation requires classification at the pixel level. Moreover, QR tag instance segmentation also has special difficulties, which are manifested because QR tags have rich textures, and even small misalignment can cause large IoU reduction. Hence it is difficult for instance segmentation of QR tag. An example of small misalignment causing large IoU reduction problem in QR tag instance segmentation is showed in Figure 1. As can be observed a QR code example image with a size of \(512\times 512\) pixels is illustrated in Figure 1, the red layer is the predicted output of a neural network (assuming that the output does not have any quality degradation, and is exactly the same as the ground truth image, and it has only misalignment) and white layer is ground truth (GT). Let variables \(d_{x}\) and \(d_{y}\) to represent the number of pixel misalignment of neural network prediction result and ground truth (GT) in the x and y directions, and set \(IoU_{dx,dy}\) to represent the corresponding IoU while \(d_{x}\) and \(d_{y}\) pixels misalignment. Figure 1a-d display \(IoU_{1,1}\), \(IoU_{15,15}\), \(IoU_{15,0}\) and \(IoU_{0,15}\), respectively. Figure 2a shows the \(IoU_{dx,dy}\) when \(d_{x}=d_{y}=1\), and can also write represented as \(IoU_{1,1}\). At this time, the offset of predicted output and GT is barely visible to human eyes. But in this case, it turns out that \(IoU_{1,1}=0.885\), and this means the IoU is decrease a lot relative to the change in \(d_{x}=d_{y}=1\). Further, the same situation can be observed for \(IoU_{15,15}\), \(IoU_{15,0}\) and \(IoU_{0,15}\). As \(d_{x}\) and \(d_{y}\) go up, \(IoU_{dx,dy}\) goes decrease very fast. Figure 2e shows the change curve of \(IoU_{dx,dy}\) while \(d_{x}\) or \(d_{y}\) changes. From the three curves, it can be seen that the value of \(IoU_{dx,dy}\) decreases rapidly with the increase of \(d_{x}\) or \(d_{y}\). Figure 1: \(Cont.\)  
From the above example, we can deduce that, due to the rich texture of the QR tag, misalignment may cause a sharp decrement of IoU. [PERSON] [29] proved that, in Faster R-CNN, quantization in the RoIPooling layer introduced misalignment between candidate region and the extracted features. Although this may not affect the classification, it has a significant negative effect on predicting pixel-accurate masks. Hence, it is necessary to find a network that has solved the misalignment problem well as the basis of our research. Mask R-CNN fixed the misalignment problem of Faster R-CNN by using RoiAlign instead of RoIPooling.Therefore, this paper mainly focuses on the research of Mask R-CNN. However, for the segmentation of QR tag instance, Mask R-CNN has the following disadvantages: it is inappropriate to use classification confidence to measure the mask quality since it only serves for distinguishing the semantic categories of proposals, and is not aware of the actual quality and completeness of the instance mask [69]. However, the quality of the instance mask is quantified as the IoU between the instance mask and its ground truth, this makes the quality of Mask R-CNN's instance mask very poor. Figure 2 shows the predicted output of Mask R-CNN model. In order to enable Mask R-CNN to predict high-quality instance mask, it is necessary to optimize IoU during training period, and also modify existing evaluation methods of the mask quality and modify mask loss. However, the problem is how can we optimize the IoU in training?