# MU R-CNN: A Two-Dimensional Code Instance Segmentation Network Based on Deep Learning
## 3 Our Contributions
### Problem Presentation and Our Solution
It is possible to directly use binary image of candidate region and its ground truth to calculate IoU. The following example will prove that this method is bad. Figure 3a shows a QR tag image with noise pollution, Figure 3b exhibits  
Figure 1: Quick response (QR) tags have rich textures, and even small misalignment can cause large IoU reduction. (**a–d**) display \(IoU_{1,1}\), \(IoU_{15,15}\), \(IoU_{15,0}\) and \(IoU_{0,15}\) respectively. Figure 2e shows the change curve of \(IoU_{dx,dy}\). The red, orange and blue curve show the change of IoU when \(d_{x}\) goes from 1 to 15, \(d_{y}\) goes from 1 to 15, and \(d_{x}=d_{y}\) goes from 1 to 15, respectively. Figure 2: The predicted output of Mask R-CNN model, the quality of mask is very poor. (**a**) and (**b**) show two examples of the Mask R-CNN’s predicting results  
a binary image of Figure 3a, and Figure 3c is its ground truth. For QR tags, the effective information part is usually printed in black. In Figure 3c, the reason why the effective information part of QR tag is shown. Generally speaking, the ground truth will be marked as white and the background marked as black in most data sets. This paper follows this general rule and marks ground truth as white to facilitate the use of this algorithm in other data sets in the future. Therefore, in the binary image showed in Figure 3b, in order to get the correct value of IoU, it is necessary to invert the gray pixel value during binarization process. The IoU value of Figure 3b,c is 0.55, where the misalignment \(d_{x}\) and \(d_{y}\) are both zero. The reduction of IoU is caused by noise pollution in candidate image, which has no relationship with misalignment. Therefore, low-quality candidate images can lower the value of IoU, disabling it from accurately measuring the completeness of the instance mask, and also making it difficult to obtain a high-quality mask image. Therefore, when calculating the IoU of candidate areas, methods should be adopted to conduct texture segmentation of the candidate image, so as to obtain high-quality texture image. Then, calculate the IoU with ground truth. In this paper, we propose to add UNet [30] branch to Mask R-CNN to segment the texture of candidate image and obtain enhanced texture information. After binarization of output image from UNet branch, we calculate the IoU value with ground truth. Here is an example to illustrate the effect of our proposed adding UNet branch method. For a noise polluted candidate image shown in Figure 4a, after a pre-trained UNet branch's prediction, the gray scale image shown in Figure 4b is obtained, and the image shown in Figure 4c is then obtained after binarization. Comparing with Figure 4b and a, it can be seen that the image has removed most influence of noise. It can also be observed that the texture of QR tag is highlighted and the texture enhancement effect is achieved. By calculating the IoU of Figure 4c and ground truth, and we get \(IoU_{0,0}=0.81\), where \(d_{x}=d_{y}=0\), and the value of \(IoU_{0,0}\) is 0.26 higher than the obtained value by using Figure 3b. Although there are still some irregular edges and incomplete details compared with ground truth. By using UNet branch texture segmentation, the corresponding IoU of low-quality image and ground truth is greatly improved. From Ref. [29] we realize that, previous researches on Mask R-CNN mainly focused on the instance segmentation of natural images. As a precise segmentation mask may not be critical in natural images, Mask R-CNN pays less attention to the quality of the fine texture mask. However, as shown in Figure 2, due to the rich texture of the QR tag, low-quality candidate images can lower the value of IoU, making it lacking the ability to accurately measure the mask quality, thus QR tag mask demands a higher level of accuracy than what is desired in natural images. Therefore, it is necessary to propose a network that can reduce the impact of low image quality on IoU calculation.