Gaussian guided IoU: A better metric for balanced learning on object detection

IET Computer VisionVolume 16, Issue 6 p. 556-566 ORIGINAL RESEARCHOpen Access Gaussian guided IoU: A better metric for balanced learning on object detection Lijun Gou, Lijun Gou State Key Laboratory of Digital Manufacturing Equipment and Technology, Huazhong University of Science and Technology, Wuhan, ChinaSearch for more papers by this authorShengkai Wu, Shengkai Wu State Key Laboratory of Digital Manufacturing Equipment and Technology, Huazhong University of Science and Technology, Wuhan, ChinaSearch for more papers by this authorJinrong Yang, Corresponding Author Jinrong Yang yangjinrong@hust.edu.cn orcid.org/0000-0002-0248-8041 State Key Laboratory of Digital Manufacturing Equipment and Technology, Huazhong University of Science and Technology, Wuhan, China Correspondence Jinrong Yang, State Key Laboratory of Digital Manufacturing Equipment and Technology, Huazhong University of Science and Technology, Wuhan, 430074, China. Email: yangjinrong@hust.edu.cnSearch for more papers by this authorHangcheng Yu, Hangcheng Yu State Key Laboratory of Digital Manufacturing Equipment and Technology, Huazhong University of Science and Technology, Wuhan, ChinaSearch for more papers by this authorXiaoping Li, Xiaoping Li orcid.org/0000-0001-9213-0416 Search for more papers by this author Lijun Gou, Lijun Gou State Key Laboratory of Digital Manufacturing Equipment and Technology, Huazhong University of Science and Technology, Wuhan, ChinaSearch for more papers by this authorShengkai Wu, Shengkai Wu State Key Laboratory of Digital Manufacturing Equipment and Technology, Huazhong University of Science and Technology, Wuhan, ChinaSearch for more papers by this authorJinrong Yang, Corresponding Author Jinrong Yang yangjinrong@hust.edu.cn orcid.org/0000-0002-0248-8041 State Key Laboratory of Digital Manufacturing Equipment and Technology, Huazhong University of Science and Technology, Wuhan, China Correspondence Jinrong Yang, State Key Laboratory of Digital Manufacturing Equipment and Technology, Huazhong University of Science and Technology, Wuhan, 430074, China. Email: yangjinrong@hust.edu.cnSearch for more papers by this authorHangcheng Yu, Hangcheng Yu State Key Laboratory of Digital Manufacturing Equipment and Technology, Huazhong University of Science and Technology, Wuhan, ChinaSearch for more papers by this authorXiaoping Li, Xiaoping Li orcid.org/0000-0001-9213-0416 Search for more papers by this author First published: 01 June 2022 https://doi.org/10.1049/cvi2.12113AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinked InRedditWechat Abstract Most anchor-based detectors use intersection over union (IoU) to assign targets to anchors during training. However, IoU did not pay enough attention to the proximity of the anchor's centre to the centre of the truth box, resulting in two issues: (1) the most slender objects were given just one anchor, resulting in insufficient supervision information for slender objects during training; (2) IoU cannot accurately represent the degree of alignment between the feature's receptive field at the anchor's centre and the object. As a result, some features with good alignment degrees are missing, while others with poor alignment degrees are used, reducing the model's localisation accuracy. To address these issues, we first created a Gaussian Guided IoU (GGIoU), which prioritises the proximity of the anchor's centre to the truth box's centre. We then proposed GGIoU-balanced learning methods, including GGIoU-guided assignment strategy and GGIoU-balanced localisation loss. This method can assign multiple anchors to each slender object, favouring features that are well-aligned with the objects during the training process. A large number of experiments show that GGIoU-balanced learning can solve the aforementioned problems and significantly improve the detection model's performance. 1 INTRODUCTION Since the advent of deep learning, object detection has evolved significantly, and numerous object detectors have been proposed. Most object detectors are divided into two types: anchor-based detectors [1-5] and anchor-free detectors [6-9]. At each place on the feature map, anchor-based detectors tile dense anchors of varying scales and aspect ratios. For anchor-based detectors, intersection over union (IoU) is adopted as a metric to assign targets for anchors during training. Because of its invariance of the box scale, IoU is the most widely used metric for measuring the similarity of two arbitrary shapes. However, IoU does not pay enough attention to the degree of closeness between the anchor's centre and the truth box's centre, and it causes two problems when using IoU to assign targets. Firstly, when IoU is used to assign targets, only one anchor can be assigned to the truth box for the majority of slender objects. RetinaNet [10] uses an IoU threshold (0.4, 0.5) to classify examples as positive or negative. When the IoU between an anchor and the closest truth box is less than 0.4, the anchor is considered a negative example. On the other hand, as a positive example, when the IoU between an anchor and the nearest truth box is greater than 0.5, the anchor is assigned to the corresponding nearest truth box. Besides, RetinaNet assigns the ground truth box with the nearest anchors to ensure there exists at least one anchor for each ground truth box. Other anchor-based detectors, such as Faster R-CNN [3] and SSD [11], employ a similar strategy to assign targets during training. Because this assignment strategy only assigns one anchor to the most slender objects, there will be insufficient supervision information to train models for slender objects, thus the model does not perform well on them. As illustrated in Figure 1a, the red solid box is a slender object, and the IoU between nearby anchors and the truth box is less than the positive threshold. Thus, when IoU is used to assign targets, only the nearest green anchor can be assigned to the truth box. FIGURE 1Open in figure viewerPowerPoint Comparison between the impact of IoU and GGIoU. The boxes with solid lines represent truth boxes and the boxes with dashed lines represent anchors. (a) The IoU between the slender object and most anchors is small, thus only the nearest anchor can be assigned to the slender object when IoU is used to assign targets. And the GGIoU is large, so multiple anchors can be assigned to the slender object when GGIoU is used to assign targets. (b) The receptive field of the feature at the purple anchor's centre aligns better with the object than that of the features at the other anchors' centres. But IoU of these three anchors is the same and cannot accurately represent the alignment degree of the features. The GGIoU of these anchors is different and can represent the alignment degree more accurately. (c) is similar to (b). GGIoU, Gaussian Guided IoU; IoU, Intersection over Union Secondly, IoU cannot accurately represent the alignment degree of the receptive field of the feature at the anchor's centre. Each anchor's centre is always attached with a feature on the feature map. And the alignment degree between the feature's receptive field and the truth box can be represented by the proximity of a feature to the truth box's centre. The closer the feature is to the truth box's centre, the better the alignment of the receptive field of the feature with the truth box. However, the IoU of anchors cannot accurately represent the alignment degree of the corresponding feature's receptive field. For example, Figure 1b, c shows that the purple anchor's centre is closer to the ground truth box's centre than the green anchor's centre. This reveals that the receptive field of the feature at the purple anchor's centre aligns better than that of the feature at the green anchor's centre. But the IoU of the purple anchor is the same as that of the green anchor. Thus, IoU fails to reveal the different alignment degrees between the purple anchor and green anchor. When IoU is used to assign targets during training, some features with a receptive field that is more aligned with the object are omitted, such as the feature at the purple anchor's centre in Figure 1c. Meanwhile, some features whose receptive field aligns worse with the object are adopted, such as the feature at the green anchor's centre in Figure 1b, which can severely hurt the model's performance. To solve these problems, we propose a better metric named Gaussian Guided IoU (GGIoU) to replace the standard IoU. We calculate the Gaussian distance between each anchor's centre and the truth box's centre, then obtain GGIoU by multiplying the Gaussian distance with the standard IoU. The Gaussian distance measures the closeness of the anchor's centre to the truth box's centre. The larger the Gaussian distance, the closer the anchor's centre to the truth box's centre. As a result, GGIoU pays greater attention to the anchor's centre's proximity to the truth box's centre than normal IoU. Additionally, we also design a GGIoU-balanced learning method, including the GGIoU-guided assignment strategy and GGIoU-balanced localisation loss. This method balances the learning process and enables the above two issues to be resolved. Extensive experiments on PASCAL VOC [12] and MS COCO [13] demonstrate that the GGIoU-balanced learning method can substantially improve the performance of object detectors, especially in the localisation accuracy. The main contribution of the paper is summarised as follows: A GGIoU is designed as a better metric by incorporating the Gaussian distance into IoU to calculate the similarity between two arbitrary shapes. A GGIoU-balanced learning method is proposed, which includes the GGIoU-guided assignment strategy and the GGIoU-balanced localisation loss to make the detector have high localisation accuracy. We conduct extensive experiments on the popular benchmarks to demonstrate that the GGIoU-balanced learning method can substantially improve the performance of popular object detectors. 2 RELATED WORK 2.1 IoU-based evaluation metric UnitBox [9] regresses the four limits of a forecast box as a complete unit using IoU loss. GIoU [14] loss is proposed to tackle the issues of gradient vanishing for non-overlapping cases, but it is still facing the problems of slow convergence and inaccurate regression. In comparison, DIoU and CIoU losses [15] are proposed to perform faster convergence and better regression accuracy. GIoU, CIoU, and DIoU are designed for localisation loss and cannot be used for target assignment. Differently, GGIoU is mainly designed for target assignment. 2.2 Target assignment strategies in anchor-based detectors Anchor-based detectors commonly utilise IoU to assign targets. RetinaNet [10] assigns the anchors with IoU below the negative threshold as negative examples and the anchors with IoU not smaller than the positive threshold as positive examples. Faster RCNN [3] and SSD [11] adopt a similar strategy. FreeAnchor [16] constructs a bag of anchors for each truth box based on the IoU and proposes a detection customised loss for learning to match the best anchors. Similarly, MAL [17] constructs bag anchors for each truth box and selects the most representative anchor based on the classification and localisation score. Besides, anchor depression is designed to perturb the features of selected anchors to decrease their confidence. ATSS [18] selects K anchors from each level of FPN for each truth box and computes IoU for these anchors. Then the IoU threshold is adaptively computed based on the IoU distribution for each truth box. Differently, we design GGIoU as an alternative and propose a GGIoU-guided assignment strategy. 2.3 Feature alignment in object detection Feature alignment is important for the performance of object detection. Multi-stage detectors [3, 19, 20] adopt RoI pooling to extract the aligned feature for each proposal. Deformable convolution [21, 22] predicts the offset for each position in the convolution kernels, and the receptive field of kernels can adaptively change based on the input feature. This makes the feature align better with the object. Guided anchoring [23] utilises the predicted box from the first stage to predict the offsets of deformable convolution, and the deformable convolution can generate well-aligned features for the second stage detector. AlignDet [24] predicts proposals at each position in the first stage and densely adopts RoIConv to extract the aligned region feature. Most of the models change the method of feature extraction to extract aligned features. Differently, the GGIoU-balanced learning method makes the training process focus more on the features close to the truth box centre, which aligns better with objects. 3 GGIoU-BALANCED LEARNING METHOD 3.1 Gaussian guided intersection over union Because of its appealing scale-invariant property, IoU is widely employed to allocate targets to anchors in anchor-based detectors. The IoU is calculated as follows: I o U = | B a ∩ B g t | | B a ∪ B g t | , $IoU=\frac{\vert {B}^{a}\cap {B}^{gt}\vert }{\vert {B}^{a}\cup {B}^{gt}\vert },$ (1)where B a = x 1 a , y 1 a , x 2 a , y 2 a ${B}^{a}=\left({x}_{1}^{a},{y}_{1}^{a},{x}_{2}^{a},{y}_{2}^{a}\right)$ are the coordinates of the top-left and bottom-right corners for the anchor box, and B g t = x 1 g t , y 1 g t , x 2 g t , y 2 g t ${B}^{gt}\,=\left({x}_{1}^{gt},{y}_{1}^{gt},{x}_{2}^{gt},{y}_{2}^{gt}\right)$ are the coordinates of the top-left and bottom-right corners for the ground truth box. As analysed above, there exist two problems when utilising the standard IoU to assign targets for the anchors during training. Algorithm 1. Gaussian Guided Intersection over Union (GGIoU) input : Parameter: β, anchor box Ba: B a = x 1 a , y 1 a , x 2 a , y 2 a ${B}^{a}=\left({x}_{1}^{a},{y}_{1}^{a},{x}_{2}^{a},{y}_{2}^{a}\right)$ , ground truth box Bgt: B g t = x 1 g t , y 1 g t , x 2 g t , y 2 g t ${B}^{gt}=\left({x}_{1}^{gt},{y}_{1}^{gt},{x}_{2}^{gt},{y}_{2}^{gt}\right)$ . Output: GGIoU. 1 For the anchor Ba, ensuring x 2 a > x 1 a ${x}_{2}^{a} > {x}_{1}^{a}$ and y 2 a > y 1 a ${y}_{2}^{a} > {y}_{1}^{a}$ : x ˆ 1 a = min x 1 a , x 2 a ${\widehat{x}}_{1}^{a}=\mathrm{min}\left({x}_{1}^{a},{x}_{2}^{a}\right)$ , x ˆ 2 a = max x 1 a , x 2 a $\quad {\widehat{x}}_{2}^{a}=\mathrm{max}\left({x}_{1}^{a},{x}_{2}^{a}\right)$ , y ˆ 1 a = min y 1 a , y 2 a $\quad {\widehat{y}}_{1}^{a}=\mathrm{min}\left({y}_{1}^{a},{y}_{2}^{a}\right)$ , y ˆ 2 a = max y 1 a , y 2 a $\quad {\widehat{y}}_{2}^{a}=\mathrm{max}\left({y}_{1}^{a},{y}_{2}^{a}\right)$ . 2 Calculating area of Bgt: S g t = x 2 g t − x 1 g t × y 2 g t − y 1 g t ${S}^{gt}=\left({x}_{2}^{gt}-{x}_{1}^{gt}\right)\times \left({y}_{2}^{gt}-{y}_{1}^{gt}\right)$ . 3 Calculating area of Ba: S a = x ˆ 2 a − x ˆ 1 a × y ˆ 2 a − y ˆ 1 a ${S}^{a}=\left({\widehat{x}}_{2}^{a}-{\widehat{x}}_{1}^{a}\right)\times \left({\widehat{y}}_{2}^{a}-{\widehat{y}}_{1}^{a}\right)$ . 4 Calculating intersection I $\mathcal{I}$ between Ba and Bgt: x 1 I = max x ˆ 1 a , x 1 g t ${x}_{1}^{\mathcal{I}}=\mathrm{max}\left({\widehat{x}}_{1}^{a},{x}_{1}^{gt}\right)$ , x 2 I = min x ˆ 2 a , x 2 g t $\quad {x}_{2}^{\mathcal{I}}=\mathrm{min}\left({\widehat{x}}_{2}^{a},{x}_{2}^{gt}\right)$ , y 1 I = max y ˆ 1 a , y 1 g t $\quad {y}_{1}^{\mathcal{I}}=\mathrm{max}\left({\widehat{y}}_{1}^{a},{y}_{1}^{gt}\right)$ , y 2 I = min y ˆ 2 a , y 2 g t $\quad {y}_{2}^{\mathcal{I}}=\mathrm{min}\left({\widehat{y}}_{2}^{a},{y}_{2}^{gt}\right)$ , w I = x 2 I − x 1 I if x 2 I > x 1 I 0 otherwise. ${w}^{\mathcal{I}}=\left\{\begin{array}{ll}\left({x}_{2}^{\mathcal{I}}-{x}_{1}^{\mathcal{I}}\right)\quad \hfill & \text{if}\quad {x}_{2}^{\mathcal{I}} > {x}_{1}^{\mathcal{I}}\hfill \\ 0\quad \hfill & \text{otherwise.}\hfill \end{array}\right.$ , h I = y 2 I − y 1 I if y 2 I > y 1 I 0 otherwise. $\quad {h}^{\mathcal{I}}=\left\{\begin{array}{ll}\left({y}_{2}^{\mathcal{I}}-{y}_{1}^{\mathcal{I}}\right)\quad \hfill & \text{if}\quad {y}_{2}^{\mathcal{I}} > {y}_{1}^{\mathcal{I}}\hfill \\ 0\quad \hfill & \text{otherwise.}\hfill \end{array}\right.$ , 5 I = w I × h I $\qquad \qquad \mathcal{I}={w}^{\mathcal{I}}\times {h}^{\mathcal{I}}$ . 6 I o U = I U $IoU=\frac{\mathcal{I}}{\mathcal{U}}$ , where U = S a + S g t − I $\mathcal{U}={S}^{a}+{S}^{gt}-\mathcal{I}$ . 7 Calculating the centre coordinates of Ba: x c a = x 1 a + x 2 a 2 ${x}_{c}^{a}=\frac{{x}_{1}^{a}+{x}_{2}^{a}}{2}$ , y c a = y 1 a + y 2 a 2 $\quad {y}_{c}^{a}=\frac{{y}_{1}^{a}+{y}_{2}^{a}}{2}$ . 8 Calculating the centre coordinates of Bgt: x c g t = x 1 g t + x 2 g t 2 ${x}_{c}^{gt}=\frac{{x}_{1}^{gt}+{x}_{2}^{gt}}{2}$ , y c g t = y 1 g t + y 2 g t 2 $\quad {y}_{c}^{gt}=\frac{{y}_{1}^{gt}+{y}_{2}^{gt}}{2}$ . 9 Calculating Gaussian variance: σ 1 = β × w I ${\sigma }_{1}=\beta \times {w}^{\mathcal{I}}$ , σ 2 = β × h I $\quad {\sigma }_{2}=\beta \times {h}^{\mathcal{I}}$ . 10 Calculating the Gaussian distance between the centres of the truth box and anchor box Dc: 11 D c = e − 1 2 x c a − x c g t 2 σ 1 2 + y c a − y c g t 2 σ 2 2 ${D}_{c}={e}^{-\frac{1}{2}\left(\frac{{\left({x}_{c}^{a}-{x}_{c}^{gt}\right)}^{2}}{{{\sigma }_{1}}^{2}}+\frac{{\left({y}_{c}^{a}-{y}_{c}^{gt}\right)}^{2}}{{{\sigma }_{2}}^{2}}\right)}$ . 12 G G I o U = I o U 1 − α D c α $GGIoU=Io{U}^{\left(1-\alpha \right)}{{D}_{c}}^{\alpha }$ . To address these issues, we develop a new metric called GGIoU to replace the conventional IoU. We compute the Gaussian distance named Dc from each anchor's centre to the truth box's centre and obtain GGIoU by multiplying the Gaussian distance with the standard IoU as Equations (2) and (4) show. D c = e − 1 2 x c a − x c g t 2 σ 1 2 + y c a − y c g t 2 σ 2 2 , ${D}_{c}={e}^{-\frac{1}{2}\left(\frac{{\left({x}_{c}^{a}-{x}_{c}^{gt}\right)}^{2}}{{{\sigma }_{1}}^{2}}+\frac{{\left({y}_{c}^{a}-{y}_{c}^{gt}\right)}^{2}}{{{\sigma }_{2}}^{2}}\right)},$ (2) σ 1 = β × w I , σ 2 = β × h I , ${\sigma }_{1}=\beta \times {w}^{\mathcal{I}},\quad {\sigma }_{2}=\beta \times {h}^{\mathcal{I}},$ (3) G G I o U = I o U 1 − α D c α , $GGIoU=Io{U}^{\left(1-\alpha \right)}{{D}_{c}}^{\alpha },$ (4) x c a , y c a $\left({x}_{c}^{a},{y}_{c}^{a}\right)$ and x c g t , y c g t $\left({x}_{c}^{gt},{y}_{c}^{gt}\right)$ represent the coordinates of the anchor box's centre and the truth box's centre, respectively. α is a hyperparameter, which is used to adjust the importance between IoU and Dc. σ1 and σ2 represent the standard deviation in the x and y directions, respectively, and they are computed by multiplying the parameter β with the width w I ${w}^{\mathcal{I}}$ and height h I ${h}^{\mathcal{I}}$ of the overlap box, respectively. The Gaussian distance Dc measures the closeness of the anchor's centre to the truth box's centre. The closer the anchor's centre is to the truth box's centre, the larger the Gaussian distance , and the range of Dc is (0, 1]. Using the overlap box's width w I ${w}^{\mathcal{I}}$ and height h I ${h}^{\mathcal{I}}$ to compute σ1 and σ2 is a crucial policy decision. Assuming that there exist two anchors whose centres are in the same position but the size of overlap boxes are different, the anchor with a larger overlap box will get a larger standard deviation and thus get a larger Dc, and vice versa. Thus, the Gaussian distance Dc not only represents the closeness of the anchor's centre to the truth box's centre but also contains the size information of the overlap box between anchors and truth boxes. How to use the width and height of the truth box to compute the standard deviations σ1 and σ2? Under this condition, the anchor with a larger overlap box has the same Dc as the anchor with a smaller overlap box, and the size information for the overlap box is missing. Thus, adopting the overlap box to compute the standard deviation is more reasonable and this will be demonstrated in the subsequent experimental results. Compared with IoU, GGIoU focuses more attention on the closeness between the anchor box's centre and the ground truth box's centre. Algorithm 1 describes the specific computation process of GGIoU. 3.2 GGIoU-guided assignment strategy To address these two issues, we develop a GGIoU-guided assignment strategy that uses GGIoU to assign targets. To begin, we calculate the GGIoU distance between all anchors and truth boxes. Then, negative examples are assigned to anchors whose GGIoU with the nearest truth box is less than the negative threshold. Positive samples are assigned to anchors whose GGIoU with the nearest truth box is greater than the positive threshold. Finally, the truth boxes are assigned to the closest anchors to ensure that at least one anchor can be assigned to each. The above problem can be solved using the GGIoU-guided assignment strategy in the following ways. First, as analysed above, only one anchor can be assigned to the truth box for most of the slender objects when the standard IoU is used to assign targets. This problem can be solved by employing a GGIoU-guided assignment strategy. GGIoU is influenced by both IoU and the Gaussian distance Dc. The Gaussian distance Dc is large for anchors close to the centre of the slender object, which can result in a larger GGIoU even when IoU is small. As shown in Figure 1a, the IoU between the anchors (dotted boxes) and the slender object (red solid box) is quite small (0.296, 0.375, 0.404, 0.429, 0.441, 0.429, 0.375, 0.167), and only the nearest green anchor can be assigned to the slender object. However, the GGIoU of these anchors is larger (0.454, 0.510, 0.529, 0.546, 0.554, 0.546, 0.510, 0.351), due to their location in the truth box's centre. When GGIoU is utilised to assign targets, multiple anchors will be assigned to this slender object and more supervision information will be provided to train the model for these slender objects. Thus, the performance of the slender objects is improved. Second, because the IoU cannot accurately represent the degree of alignment between the anchor's central feature and the truth box, some features are poorly aligned with the object while others that are well aligned with the object are missing during the training process. This problem can be alleviated by the GGIoU-guided assignment strategy. As shown in Figure 1b, the IoU of the purple anchor is the same as the IoU of the green anchor, and they all can be assigned to the truth box. Obviously, the receptive field of the feature at the purple anchor's centre aligns better than that of the features at the green anchor's centre. Therefore, IoU cannot accurately represent the alignment degree of the feature. So when using IoU for target allocation, the feature at the green anchor's centre with the worst alignment degree of the object is also used to train the object. But when using GGIoU, the GGIoU of the green anchor is smaller than IoU, and the green anchor is discarded. When IoU is used to assign targets, both the green and purple anchors are assigned to be negative examples, and the feature at the purple anchor's centre that aligns well with the object is missing, as shown in Figure 1c. And when GGIoU is used, the central purple anchor is assigned to be a positive example and the corresponding feature is adopted for training this object. In summary, the GGIoU-guided assignment method biases the training process towards features that match well with objects (the purple anchor in Figure 1c), while eliminating features that align poorly with objects (the green anchor in Figure 1b). There still exists a problem. Figure 1b shows that the feature at the blue anchor's centre aligns worse than the feature at the purple anchor's centre. The blue anchor can be assigned to be a positive example regardless of whether IoU or GGIoU is used, and the feature at the purple anchor's centre is considered the same as the feature at the purple anchor's centre. Can we bias the model more to the features aligning better with the objects? Inspired by IoU-balanced localisation loss [25], we design GGIoU-balanced localisation loss to realise this goal. 3.3 GGIoU-balanced localisation loss Under the framework of the traditional training method, all target samples are equally treated by sharing the same weight in the localisation loss function. To guide the model and pay more attention to the features aligning better with the objects, we proposed GGIoU-balanced localisation loss as Equations (5) and (6) shows: L loc = ∑ i ∈ P o s N ∑ m ∈ c x , c y , w , h w i G G I o U i ∗ smoot h L 1 l i m − g ˆ i m ${L}_{\mathit{loc}}=\sum\limits _{i\in Pos}^{N}\sum\limits _{m\in cx,cy,w,h}{w}_{i}\left(GGIo{U}_{i}\right)\ast \text{smoot}{\mathrm{h}}_{L1}\left({l}_{i}^{m}-{\widehat{g}}_{i}^{m}\right)$ (5) w i G G I o U i = w loc ∗ G G I o U i λ ${w}_{i}\left(GGIo{U}_{i}\right)={w}_{\mathit{loc}}\ast GGIo{U}_{i}^{\lambda }$ (6)where l i c x , l i c y , l i w , l i h $\left({l}_{i}^{cx},{l}_{i}^{cy},{l}_{i}^{w},{l}_{i}^{h}\right)$ represent the encoded coordinates of the prediction box and g ˆ i c x , g ˆ i c y , g ˆ i w , g ˆ i h $\left({\widehat{g}}_{i}^{cx},{\widehat{g}}_{i}^{cy},{\widehat{g}}_{i}^{w},{\widehat{g}}_{i}^{h}\right)$ represent the encoded coordinates of truth boxes, which is the same decoding style as that of Faster R-CNN [3]. wloc represents the weight of localisation loss, which is manually adjusted to ensure that the loss in the first iteration is the same as the baseline counterpart. λ is a hyperparameter, which is used to adjust the weight of GGIOU in the loss. When utilising the GGIoU-balanced localisation function, there are two advantages: (1) GGIoU-balanced localisation loss can suppress the gradient of outliers during training; and (2) GGIoU-balanced localisation loss make the training process pay more attention to the features aligning better with objects. 4 EXPERIMENTAL RESULTS 4.1 Experiment setting 4.1.1 Data set All detection baselines are trained and evaluated on three object detection data sets including MS COCO [13], PASCAL VOC [12] and SCD [26]. MS COCO contains about 118K images for training (train-2017), 5k images for validation (val-2017) and 20k for testing (test-dev). For PASCAL VOC, the VOC2007 includes 5011 training images (VOC2007 trainval) and 4952 testing images (VOC2007 test) while VOC2012 contains 11,540 training images and 10,591 testing images. The combination of VOC2007 trainval and VOC2012 trainval is adopted to train models while VOC2007 (VOC2007 test) is used to evaluate models. SCD [26, 27] is a dense carton detection and segmentation data set and contains LSCD and OSCD subsets. We conduct experiments on LSCD, which contains 6735 images for training and 1000 images for testing. 4.1.2 Evaluation protocol We adopt the same evaluation metric as the MS COCO 2017 Challenge [13]. This includes AP (averaged mAP across different IoU thresholds IoU = {0.5, 0.55, …, 0.95}), AP50 (AP at IoU threshold of 0.5), AP75 (AP at IoU threshold of 0.75), APS (AP for small scales), APM (AP for medium scales) and APL (AP for large scales). 4.1.3 Implementation details All experiments are implemented by using PyTorch and MMDetection [28]. All models are trained on 4 NVIDIA GeForce 2080Ti GPUs while the learning rate changes linearly based on the mini-batch [29]. For all ablation studies, RetinaNet with ResNet18 as the backbone is trained on train-2017 and evaluated on val-2017 with the image scale of [800, 1333]. For all baseline models, we follow the default settings of MMDetection. 4.2 Ablation studies on GGIoU used as sampling metric 4.2.1 Ablating the style of calculating standard deviation of Gaussian distance As shown in Table 1, when using the width and height of truth boxes to export the standard deviations σ1 and σ2 of Gaussian distances, the sampling strategy based on GGIoU can improve the model's AP by up to 0.5%. And when the overlapping area between ground truth boxes and prediction boxes acts as the base size to calculate standard deviations, the mAP is 0.9% higher than the baseline, and it is also 0.4% higher than that using the width and height of truth boxes to export the standard deviation, as shown in Table 2. These results echo the argument in Section 3.1 and verify that employing the shape of the overlapping area between the truth boxes and the prediction boxes not only measures the distance of two boxes but also reflects the overlapping scale, which is reasonable. TABLE 1. The results of