## 4 Research Method
### Improved Faster R-CNN with Multi-Scale Attention Mechanism
#### 4.3.3 Improved ROI-Alignment  
The ROI-Pooling layer was originally introduced in Fast R-CNN and has since found widespread application in Faster R-CNN. In the Faster R-CNN framework, the RPN network generates candidate boxes of diverse sizes. Subsequently, in the ROI-Pooling  
Figure 8: Network structure of coordinate attention (**left**) and multi-scale attention (**right**).  
layer, feature pooling is executed to map candidate boxes of various sizes onto fixed-size feature maps. ROI-Pooling incorporates two rounding quantizations. The first quantization converts the floating-point coordinates of candidate boxes into integers, while the second quantization further divides the quantized candidate regions into N \(\times\) N equally-sized sub-regions, rounding the edges of each sub-region, as illustrated in Figure 9. However, ROI-Pooling suffers from a drawback where the two rounding operations lead to an approximation of the region's value and varying sizes of each region. This issue can result in position errors when measuring boxes, consequently diminishing detection accuracy, especially for smaller targets. Given the focus of this paper on detecting errors in GUI interface display, some of which entail small geometric areas and occupy fewer image pixels, we address this limitation by employing ROI-Align as an enhancement. This improves the network's generalization capability and detection accuracy, as depicted in Figure 9.  
ROI-Align, introduced in Mask RCNN [37], addresses the quantization errors inherent in ROI-Pooling. In contrast to ROI-Pooling, ROI-Align retains the original floating-point values, eliminating the need for quantization. Bilinear interpolation is employed to calculate pixel values based on floating-point image coordinates, ensuring precise alignment between the pixels of the original image and the feature map (Figure 10).  
The bilinear interpolation algorithm, as shown in Figure 7, first performs linear interpolation between A11 and A21 to obtain Q1. Similarly, Q2 can be obtained by performing linear interpolation between A12 and A22. Q1 and Q2 can be calculated using Equation (5).  
\[\begin{split} f(Q_{1})&\approx\frac{x_{2}-x}{x_{2}- x_{1}}f(A_{11})+\frac{x-x_{1}}{x_{2}-x_{1}}f(A_{21})\text{ Where }Q_{1}=(x,y_{1})\\ f(Q_{2})&\approx\frac{x_{2}-x}{x_{2}-x_{1}}f(A_{12} )+\frac{x-x_{1}}{x_{2}-x_{1}}f(A_{22})\text{ Where }Q_{1}=(x,y_{2})\end{split} \tag{5}\]  
Figure 10: Example of bilinear interpolation.  
Figure 9: Example of ROI-Pooling (**left**) and ROI-Align (**right**).  
Then, linear interpolation is performed between Q1 and Q2 in the y direction with Equation (6).  
\[f(P)\approx\frac{y_{2}-y}{y_{2}-y_{1}}f(Q_{1})+\frac{y-y_{1}}{y_{2}-y_{1}}f(Q_{2}) \tag{6}\]  
Finally, the final position is obtained through Equation (7).  
\[\begin{split} f(x,y)&\approx\frac{(x_{2}-x)(y_{2}-y )}{(x_{2}-x_{1})(y_{2}-y_{1})}f(A_{11})+\frac{(x_{2}-x)(y-y_{1})}{(x_{2}-x_{1}) (y_{2}-y_{1})}f(A_{12})\\ &+\frac{(x-x_{1})(y_{2}-y)}{(x_{2}-x_{1})(y_{2}-y_{1})}f(A_{21})+ \frac{(x-x_{1})(y-y_{1})}{(x_{2}-x_{1})(y_{2}-y_{1})}f(A_{22})\end{split} \tag{7}\]  
As illustrated in Figure 9, the dashed box represents the feature map, while the solid box signifies an ROI feature (with a size of \(2\times 2\)) featuring four sampling points per sub-region. ROI-Align utilizes bilinear interpolation to estimate pixel values corresponding to these grid points and subsequently employs max pooling or average pooling for each sub-region to yield a \(2\times 2\) output result.