## 3 Instance Segmentation Framework Based on Improved YOLACT
### Improved Bounding Box Loss Function for Redundant Boxes
\(w_{c}\) represents the width of the minimum bounding rectangle of two boxes, \(h_{c}\) represents the height of the minimum bounding rectangle, and the sum of the squares of these two is also the square of the diagonal length of the minimum bounding rectangle. \(w^{st}\) and \(h^{st}\) represent the width and height of the ground truth box, while \(w^{p}\) and \(h^{p}\) represent the width and height of the predicted box, respectively. It can be seen that the value of the RIoU Loss may be greater than 1, but it should be limited to between 0 and 1 so that it can allow the RIoU function to work normally. The limitation for RIoU Loss is as follows:  
\[\text{RIoU Loss}= \begin{cases}1,&\quad if\ \text{RIoU Loss}>1\\ \text{$RIoU_{Loss}$},&\quad if\ \text{RIoU Loss}\leq 1\end{cases}  \tag{11}\]  
As shown in Equation (9), the proposed loss function consists of three parts. The left item is a penalty term for small prediction boxes. If the box is smaller relative to the ground truth box, it is more likely to be suppressed. When the IoU reaches a larger value, the loss value will be smaller, while when IoU reaches a smaller value, the loss value will rapidly increase. The middle term is a penalty term for the degree of deviation from the center of the prediction box. The farther the prediction box deviates from the center of the ground truth box, the easier it is to suppress it. The last item is a penalty term for shape similarity. When the aspect ratio of the predicted box differs significantly from the ground truth box, it indicates that the shape of the predicted box is incorrect and does not meet the requirements of the model. When the IoU of the prediction box is small, the loss value of the area penalty term is already large, which means that the quality of the prediction box is considered poor. When the IoU is large, we pay more attention to the center deviation and shape deviation. Hence, these two penalty terms are multiplied by the IoU and IoU\({}^{2}\), respectively, to enhance their effectiveness, particularly when the IoU is large. The three penalty terms proposed in this paper share a common principle: their values will rapidly increase with the degree of deviation from the correct box, and when the IoU is small, the penalty for the IoU is the main term. When the IoU is large, center deviation and shape deviation become the main penalty terms. This design aims to effectively minimize redundant target boxes and prevent the algorithmic misdetection of targets. Figure 6 shows a comparison of the RIoU Loss with the IoU Loss and GIoU Loss in three scenarios, with the red box representing the ground truth box and the green box representing the predicted box. The IoU of the green box and the red box in Figure 6 is 0.45. The predicted boxes in (a) and (b) have the same shape and are similar rectangles to the ground truth box. While (a) has no center deviation, (b) deviates from the center of the ground truth box. Furthermore, (c) has no center deviation, but its aspect ratio is different from the ground truth box. Although the IoUs are both 0.45, the center deviation and shape deviation bring about additional penalties when calculating the RIoU loss function. The IoU Loss and GIoU Loss do not take into account the distance that the box deviates from the ground truth box, nor do they consider the shape of the predicted box. Therefore, it can be seen that neither the IoU loss function nor the GIoU loss function can effectively evaluate the quality of the box.