$\alpha$-IoU: A Family of Power Intersection over Union Losses for Bounding Box Regression
Authors: Jiabo He
Sarah Monazam Erfani
Xingjun Ma
James Bailey
Ying Chi
Xian-Sheng Hua
Venue: NeurIPS 2021 Poster (2021)
Keywords: bounding box regression
localization loss
object detection
intersection over union

Abstract: Bounding box (bbox) regression is a fundamental task in computer vision. So far, the most commonly used loss functions for bbox regression are the Intersection over Union (IoU) loss and its variants. In this paper, we generalize existing IoU-based losses to a new family of power IoU losses that have a power IoU term and an additional power regularization term with a single power parameter $\alpha$. We call this new family of losses the $\alpha$-IoU losses and analyze properties such as order preservingness and loss/gradient reweighting. Experiments on multiple object detection benchmarks and models demonstrate that $\alpha$-IoU losses, 1) can surpass existing IoU-based losses by a noticeable performance margin; 2) offer detectors more flexibility in achieving different levels of bbox regression accuracy by modulating $\alpha$; and 3) are more robust to small datasets and noisy bboxes.

Alpha-loU: A Family of Power Intersection over Union Losses for Bounding Box Regression
Jiabo Hel. Sarah Erfani', Xingjun Ma² James Bailey', Ying Chi Xian-Sheng Hua
'School of Computing and Information Systems, The University of Melbourne School of Computer Science, Fudan University
DAMO Academy, Alibaba Group (jiaboh@student. sarah. verah.erfanil baileyj@/unimelb edu. danxjma@gmaii com, (xinyi.cy cy, xiansheng. hxs) Saaeeaaaaaaaaaaaaaeaaeaaaaaaaaaa wwwaaannaaaaaaaaaaaaaaaaaaaaaaaa com
Abstract
Bounding box (bbox) regression is a fundamental task in computer vision. So far, the most commonly used loss functions for bbox regression the Intersection over Union (loU) loss and its variants. In this paper, we generalize existing loU- family of power IoU losses that have power IoU term and an additional power regularization term with a single power parameter a. We call this new family of losses the a-IoU losses and analyze properties such as order preservingness and loss/gradient reweighting. Experiments on multiple object detection benchmarks and models demonstrate that a-IoU losses, 1) surpass existing IoU-basee losses by noticeable performance margin; 2) offer detectors more flexibility in achieving different levels of bbox regression accuracy by modulating o; and 3) are more robust to small datasets and noisy bboxes.
based losses
Introduction
Bounding box (bbox) regression localizes an object in an image/video by predicting bbox for the object, which is fundamental to object detection, localization, and tracking. For example, the most advanced object detectors often consist of bbox regression branch and classification branch with the bbox regression branch generating bboxes to localize objects for classification. In this work, we explore more effective loss functions for bbox regression in the context of object detection.
Whilst early works object detection use -nom losses [ for bbox regression, recent works directly adopt the localization performance metric, i.e., Intersection over Union (IoU), the local- ization loss (28 39). Compared with -n-or losses, the IoU loss invariant to bbox scales, thus helping train better detectors. However, the IoU loss suffers from the gradient vanishing problem when the predicted bboxes are not overlapping with the ground truth, which tends to slow down convergence and result in inaccurate detectors. This has motivated the design of several improved loU-based losses including Generalized loU (GloU), Distance-loU (DIoU) and Complete loU (CloU). GloU introduces a penalty term into the IoU loss to alleviate the gradient vanishing problem [32) while DIoU and CloU consider the central point distance and aspect ratio between predicted bboxes and their ground truth in penalty terms [43]
this paper, we present new family of IoU losses obtained by applying power transformations to existing loU-based losses. We first apply the Box-Cox transformation 2 to the loU loss Lio 1- IoU and generalize it to power IoU loss: La-toU = (1 - IoU )/a, a > 0, denoted as a-loU. We further simplify a-IoU to La-toU = 1 I for 0 and extend to more general form
35th Conference Neural Information Processing Systems (NeurIPS 2021).
'This research was done when Jiabo He interned with the DAMO Academy, Alibaba Group. t Cooresponding authors.

---

with an additional power regularization term (see equation (3)) This allows us to generalize existing loU-based losses, including GloU, DIoU, and CloU, to a new family of power IoU losses (see equation (4) for more accurate bbox regression as well as object detection.
We show that, relative to LioU, loU with a > 1 up-weights both the loss and gradient of high loU objects, leading to improved bbox regression accuracy. When 0 < a < 1, it down-weights high IoU objects which we find hurts regression accuracy. The power parameter a can serve as knob to adapt a-loU losses to meeting different levels of bbox regression accuracy (precision measured under different IoU thresholds). with a > 1 for high regression aceuracy (i.e., high IoU thresholds) by focusing more on those high IoU objects. We also empirically show that a is not overly sensitive to different models or datasets, with a = 3 performing consistently well in most cases. The family of a-loU losses can be easily applied for improving state-of state-of-the-art detectors under both clean and noisy bbox settings without introducing additional parameters to these models (making any modifications to training algorithms), nor increasing their training/inference time.
In summary, our main contributions are follows:
We propose new family of power IoU losses called I-U for accurate bbox regression and object detection. a-IoU presents a unified power generalization of existing loU-based losses.
We analyze a set of properties of a-IoU, including order preservingness and loss/gradient reweight- ing, to show that a proper choice of a (i.e., a > 1) can help improve bbox regression accuracy by adaptively up- up-weithting the loss and gradient of high IoU objects.
We empirically show, on multiple benchmark object detection datasets and models, that a-IoU losses can consistently outperform existing loU-based losses and provide more robustness for small datasets and noisy bboxes.
2 Related Work
Object Detection Models. There exist two mainstream types of detection models: anchor-based and anchor-free detectors. Anchor-basee detectors can be further divided into two-stage and one- stage models. Two-stage unchor-based detectors (c.g., R-CNN series [I] 31 14 3], HTC (5). and TSD 33) are firstly proposed in object detection tasks, which are composed of region proposal networks (RPNs) and classifiers. RPNs generate large number of foreground and background region proposals, followed by networks to classify objects in the proposals. Towards real-time object detection, one-stage unchor-bases detectors (e.g., YOLO series [29 30 I. RetinaNet [21]. and SSD
(24 are developed to predict bboxes and categories at the same time, thus no longer need RPNs. Anchor boxes with prior scales and aspect ratios should be defined before training anchor-based detectors. Techniques have been proposed to mitigate the sensitivity of these models to hand-picked anchor boxes, for example, attention- fusion networks 33 and clustering algorithms [30). These techniques learn prior anchors from the training set for every sliding window or grid cell.
Recently, anchor-free detectors such as CornerNet [16). CenterNet, (8] ExtremeNet [45), and CentripetalNet [7]. have also been proposed to get rid of anchor priors. These models first predict locations of keypoints (corners, centroids, or extreme points), then group them into the same bboxes if they are geometrically aligned. There also exist other models that generate pixel-wise results. For example, CenterNet, estimates pixel-level categories of objects along with their sizes and offsets 44). FCOS generates pixel-wise classification, centerness, and bbox (top, down, left, right) results using multi-head CNNs (34), followed by the Adaptive Training Sample Selection (ATSS) 40 as an improvement on automatically selecting positive and negative samples. In addition, transformers (c.g., DETR series 4 (66) have also been developed for object detection without anchor generation par with the above CNN-based
or non-maximum suppression (NMS), achieving the performance detectors. In this work, we propose a new family of generalized IoU losses to improve the performance of these detectors without any architectural modifications, which is orthogonal to the above research.
Bounding Box Regression Losses. anchor-based detectors regress offsets between ground-trute bboxes and their closest anchors, while anchor-free detectors predict keypoints of objects with some frameworks also generating the sizes of the bboxes. The predicted offsets or keypoints (w/ or w/o bbox sizes) are then mapped back to the pixel space for generating the bboxes. Localization losses usually compare the generated bboxes with their ground truth. Early works adopt -o-mo losses (II] for bbox regression, which have been found sensitive to varying bbox scales. Recent works replace

---

(X Y). the task space to the annotation space f X + Y. In object detection, each y; = (ct,k) e B.,,) yo, k= where m. them with the IoU loss and its variants such as BloU, GloU, DIoU and CloU for bbox regression, as IoU the metric for localization and it is scale-invariant 28 39 The Bounded IoU (BIoU) loss maximizes the loU overlap between the region of interest (Rol) and the ground truth based on set of ToU upper bounds [35] GloU is proposed address the problem of gradient vanishing on non-overlapping examples, which are examples having non-overlapping predicted bboxes with the ground truth (IoU is zero) [32]. DIoU and CloU (43) losses further consider the overlapping area, central point distance, and aspect ratio in IoU and the regularization terms. These regularization terms can help improve the convergence speed as well as the final detection performance. There are also losses designed to focus more on high IoU objects, for example, the Rectified IoU (RIoU) loss [36], and the Focal and Efficient loU (Focal-EloU) loss [41]. These loss functions increase gradients of those examples that are in high bbox regression accuracy. However, RIoU and Focal- EooU are neither concise nor generalized compared with other loU-based losses. In this paper, we apply a power transformation to generalize the above vanilla IoU loss and regularized loU-based losses for both their loU and regularization terms. The new family of generalized losses improve bbox regression accuracy by adaptively reweighting the loss and gradient of high and low IoU objects.
There are also works on AutoML-based loss function search for computer vision tasks 23 [8 17 Despite their advantage in saving human efforts, these methods are very expensive in searching qualified loss functions (e.g., days of searching time on multiple GPUs), and probably with limited performance improvement based on existing losses [23). We will empirically compare with one of these losses in our experiments.
3 a-IoU Losses for Bounding Box Regression
3.1 Preliminaries
We study the problem of bbox regression in object detection. Let X E RRd be the input space and Y RR, be the annotation space, with d, and dy denoting the input and annotation dimensions, respectively. Given a dataset D = {( y:)} of n training examples with each (æ., y:) E learn a function f (represented by a detector network) that maps the input
is the total number of objects in æ; Ci,e is the category of the 1tth object in æ; and B, is its bbox.
The bbox regression performance is measured by the Intersection over Union (loU) metric between the predicted bbox B and the ground truth 8 IoU = / B Positive examples (both true and false positives) are determined from the set of predictions according to IoU threshold, based on which the Average Precision (AP) over all categories of objects can be calculated. E.g.. AP50 measures the AP of objects localized by bboxes with an IoU that is above the threshold 0.5. The final performance of detector is commonly evaluated by the mean Average Precision (mAP) across multiple IoU thresholds. For instance, the popular metric mAP measures the mAP of examples across the set of IoU thresholds ranging from 0.5 to 0.95 with a stride of 0.05.
3.2
I- Losses
The vanilla IoU loss is defined as Lis 1 ToU. and generalize the loU loss to an a-loU loss:
We first apply the Box-Cox transformation
1 1 Io
Ca-1oU
a a >
By modulating the parameter a in a-loU, one can derive most of the IoU terms in existing losses, e.g., log( (og((o)),, ), Io and loU2, When a - O. we obtain lima-+0 Ca-toU = log( = LiggtoU (see the proof in Appendix(A). We recover the loU loss with a 1: LI-WUU
IoU LIoU- And C2 = ] (1 - 2 = 5 Liau, when a = 2. We can also extend the above I- formula to
39
loss functions with multiple IoÚ terms (e.g. RIoU (36 by using multiple a values.
"The Box-Cox transformation has also been applied for generalizing the Cross Entropy (CE) loss into the Generalized Cross Entropy (GCE) loss 42]. Such a generalization unifies the Mean Absolute Error (MAE) and the CE loss, increasing the robustness to noisy labels for classification tasks. However, GCE may slow down convergence and lead to degraded performance [37 25). In contrast, our a-IoU generalization is for bbox regression and can improve the localization performance both clean and noisy bbox datasets.
3

---

High loU examples
High laU examples
0.8
Gradientt
3
Loss
0.2
1
0
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
IoU
loU
.x-0.5
m=1 m=1.
o==
2=2.5
0=3
0055
6=1
#=1.5
a=2
2=5.
rr=3
Figure 1: Correlation between IoU and Lo- 1 - Io (left) and its absolute gradient IVou (right) with different a E [0.5,3], According both plots. Lo- reweights all objects adaptively and distinctively for 0 < < 1ss. a (a= marks the IoU loss).
We simplify the above a-IoU formula for > 0 and + as this case, the denominator a in equation ①1 is just positive constant in the objective. This gives us two cases of the a-loU loss for -> and -> respectively:
La-IoU = { (oog([ToU) -- 0,
( IoUO a+ O.
(2)
Here, more interested in the 2 1. We then extend the above a-IoU loss for a + to a more general form by introducing power penaltyyfeegularization term into the formula:
O as most state-of-the-Bn IoU-based losses have an
La = TOU + pod (B, Bot),
where 0 > 0, a2 > 0, and p (B, Bst) denotes any penalty term computed based on B and Bst, This simple extension allows straightforward generalization of existing IoU-based losses their a-IoU versions. In Appendix I.2.I we empirically show that Lo- is not sensitive to a22. We thus maintain the power consistency between the IoU term and the penalty term and take a1 02 as simple choice when training the detectors.
With the above loU formula, LIuU, LGoU LDOUU, and LCIoU using the same power parameter a for the loU and penalty terms:
now generalize the commonly used IoU-based losses including
LioU IoU = La-IoU = - IoUa
IC |C\(BU (BU B
La-GioU = 1 L + CV (BUBB B
C
CGIOU = IoU +
|C|
|C|
p² bor)
Da.DIoU = LoU + 2 (2 bot)
LDIOU = ToU +
c²
ca0
LCIU = 1 ToU + (b, b
(b, bat
+ 3r Ca.
ToU +
(Bu)a.
c²
220
where C in LaNO denotes the smallest convex shape enclosing B and B and ae in LDIOU denote central points of B and Bat with p(-) being the Euclidean distance and being the diagonal length of the smallest enclosing box; and in LCioU - (arctan / arctan w) give us the family of power IoU losses for bbox regression with their original versions recovered at = 1. Note that the above a-loU generalization can be easily extended to more complex loss functions that have multiple loU or penalty terms (e.g., Clat). Next, we will analyze the properties of a-loU losses when a takes different values.
T They
3.3 Properties of a-loU Losses
Here, we focus on the vanilla a-loU formula L.-IoU I0U to analyze properties, as penalty terms may affect these properties differently. Figure[iillustrate the correlation between loU and LoU (left) and the magnitude of its gradient w.r.t. ToU, i.e., Viouaa (right). One key observation is that the IoU loss (i.c.. 1) has a linear correlation with IoU and the gradient

---

is a constant, while La- reweights objects adaptively (according to their IoU values) following different reweighting schemes with < versus a 1.
The power transformation in LoU preserves key properties of LioU as a performance metric, includ- ing non-negativity, identity of indisceribless, symmetry, and triangle inequality [32]. Furthermore, we analyze the following important properties of La-toU with detailed derivations deferred to Appendix We first let B; and B3 be two predicted bboxes by two different models M, and M, respectively, and B; and B, correspond to the same ground truth Bst with IoU (B., BBy) < (B, Bot). Then we have the first property of La-ooU
Property 1 (Order Preservingness). fa-ooU preserves the orders of both loU and Liou: (B., B < [o, B5t) = LIou(B,, Bot) > Liou(Bj, B9t) = Lastoul B9t) La-tov Bj, Bot).
The above property indicates that both La- and LioU are monotonically decreasing functions w.r.t. IOU. As Lo- preserves the order of LiaU strictly, it is guaranteed that arg min min (B, B is identical to arg max g IlU (B, Bst) and arg mingCwv B, 3. In other words, the optimal solution arg max y g/o (B, Bsy) can be obtained by minimizing either Lo--oll or LIoU. Following this, the adaptive relative loss reweighting scheme of La-ooU can be characterized by the second property:
Property 2 (Relative Loss Reweighting). Compared with LioUs La- adaptively reweights the relative loss of all objects by wc, = Lo- Liot = 1+ (IoU- - // 0) == 1, and lim 1oU--1 Wc. = a.
with we, (ToU' (I.(loU' =
The second property indicates that La-tou will adaptively down-weight and up- weight the relative loss of all objects according to their IoUs when 0 < < 1 and a > 1, respectively. We further note that, when a > 1, the reweighting factor wc. increases monotonically with the increase of loU (w.. grows from 1 to a') while decreasing monotonically with the increase of loU when 0 < a < 1 (we.. decays from 1 to a:). We will empirically show that the up- up-weighting scheme of La with a > 1 can help the model focus more on high IoU objects improve both the localization (i.e., predict more high IoU objects) and detection (i.e., more accurate at high APs) performance]] Similarly, we can obtain the third property of adaptive relative gradient reweighting owned by Ca-LoU as follows:
Property 3 (Relative Gradient Reweighting). Compared with LioUs Lo- adaptively reweights the relative gradient of all objects by wv. = Woouaatoos V VouLl = aIooo0-11, with the turning point at IoU = =- E (0, !) when 0 < a < and IoU = T E (2. when a > 1.
When a > the above reweighting factor wv, increases monotonically with the increase of IoU, while decreasing monotonically with the increase of loU when 0 < a < 1. This relative gradient reweighting scheme is also adaptive to IoU, with the turning point from up up-weighting to down- weightiig at IoU- = aa-aa E (0, 2) when 0 < a < 1, and from down-weightig to up-
weighting at IoU = - E (c, when a > 1. The gradient reweighting scheme is bounded by wv = 1)
i.e., 0 < wv S a when a > 1, and wv, 2 when 0 < a < 1. This relative gradient reweighting scheme allows the model to learn objects with adaptive speeds (i.e.. different gradients) according to their IoUs. Theoretically, when
2. V > ViauCioul for ToU E (0.5, 1]. which accelerates the learning of all positive IoU objects at AP However, we empirically show that a-loU losses with a = 3 perform more competitively than those with a = 2 in most cases. It is probable that a-IoU losses with a = 3 further up up-weight the relative loss of objects with LoU E (0.5, 1J. although a-I losses with = 2 also beat existing basclines (see Figure 6). This property is both data-agnostic and model-agnostic, so we recommend a = 3 or a E [2, in practical use for other datasets and models.
The above loss and gradient reweighting schemes can also be inferred from Figure I with detailed proofs in Appendix A To summarize, La-toU trains better detectors than LIo for the following reasons. First, the same optimal loU can be achieved by Lo as that by LisU (Property 1). Second, Loo with a > 1 focuses more on high IoU objects by up-weighting their relative loss (Property
"The relative loss reweighting scheme of La-bo is reminiscent of the reweighting scheme of the focal loss FL( - (1 ") log(p). 7 O, which was proposed encourage the learning of low confidence foreground objects the presence of large number of high confidence backgrounds 2I]. In contrast, La-i with a > I focuses more on high IoU objects as the final performance is comprehensively measured by mAPs0.ss. with low IoU objects (here ToU < 0.5) suppressed at evaluation. Note that focal loss is usually designed for the classification branch of object detectors while our alloU is for the bbox regression branch.
5

---

2). Third, LoU with a > 1 helps detectors learn faster on high loU objects (here ToU E (aT=o, ID) through up-weighting their relative gradient (Property 3). In Appendix [A] we also provide an analysis of the absolute loss and gradient reweighting properties (Property 4 and 5), showing the additions of a-IoU to loU. Specifically, when a > 1, La-toU adds an absolute loss weight to LioU (i.e.. Wc. La-toU LioU LoU IoU > 0 for LoU E (0, 1)), which creates more space for optimization on all levels of objects (Property 4). Likewise, La-loU puts an absolute gradient aloUa-1
weight for high IoU objects (i.e., wv. |VouLa-toul - Wiwl
> 0 for IoU E (a--a, 1) such that the lcarning of high IoU objects is accelerated (Property 5). Both of the absolute and relative properties of La-IoU are adaptive to the IoU values of the objects. Such reweighting schemes will provide more flexibility in achieving different levels of bbox regression accuracies (AP measured under different IoU thresholds).
Learning Dynamics of La- Training with Ca-toU is a dynamic process and should be interpreted based on both the absolute and relative properties. With a > 1, easy examples will be learned first with increasing speed towards IoU = 1, while hard examples will be learned gradually and accelerated later on as their IoU improves. We will empirically show in Figure that up-weighting the loss and gradient of high IoU objects can boost the training at the later stage. As a comparison, we will also show that a-IoU losses with 0 < a < 1 tend to degrade the final performance in Section Reducing the loss and gradient of high IoU objects ends up with more poorly localized objects.
4.4
4 Experiments
4.1 Datasets and Training Setup
We conduct all experiments on two popular henchmarks, i.e., PASCAL VOC [9] and MS COCO 22 On the PASCAL VOC benchmark, we train all models on the trainval set 2007+2012 (containing 16, 551 images from 20 categories) and evaluate them on the test set 2007 (containing 4, 952 images) 29. On the MS COCO benchmark, we train all models on the training set 2017 (containing 118K images from 80 categories) and evaluate them on the val set 2017 (containing 5K images) [22]. We train all state-of-the-art models with the original implementation released by the authors. Specifically, we follow the original implementation' 's training protocol with default parameters and the number of training epochs with different losses [3] 32 43 4). Implementation details of all models are
given in Appendix [B.]] All experiments are run with NVIDIA V V100 GPUs. Code is available at https://gtt wtt.bb..gttuu.t.//gg com/ aom/Jaaobb99/A1pha IoU
4.2 Results and Analysis
We first validate the effectiveness of a-IoU losses in training both anchor-based and anchor-free models on the two datasets. We choose YOLOv5s (i.c., YOLOVS small) and YOLOv5x (i.c., YOLOv5 extra large) as one-stage anchor-based models, and DETR (ResNet-50) as an anchor-free model. Both a-IoU losses (i.c., Ca-toU and Ca-Diou) are generalized from existing baselines following equation (4). From Table T. we can observe that a-U losses surpass existing losses consistently across multiple models and datasets in terms of both mAP and mAP75.95. especially at the high bbox regression accuracy mAP73:93. The superiority of a-IoU losses is more pronounced at high accuracy levels, which might reach more than 60% relative improvement at AP Interestingly, a-IoU losses tend to help more of light models (e.g., YOLOv5s with 7.3M parameters and 17 GFLOPs) than heavy models (e.g., YOLOVV with 87. 7..7M parameters and 218.8 GFL.Pss.. This indicates that a- losses hold more advantage while training light models in computing-resouse -imoourrre-mmiteed scenarios, such mobile devices, autonomous vehicles, and robots.
The consistent improvements on both PASCAL VOC and MS COCO demonstrate the stability of a-IoU losses across different datasets. In addition, we also verify its robustness to extremely small training sets in Appendix B.2.2] where a-IoU losses beat existing losses at various scales, i.e., from 4K (25% trainval set of PASCAL VOC 2007+2012) to 1 118K (the entire training set of MS COCO 2017) samples. It is possible that a-U losses may not perform well if measured by a single low AP metric. For example, there may be less than 0.5% performance drop at AP5o when = 3, however, this is compensated by the significant boost at high APs. With some examples from the test set of PASCAL VOC 2007 (Figure and the val set of MS COCO 2017 (Figure we show that a-loU

---

Table 1: The performance of YOLOv5s, YOLOv5x and DETR models trained using different localization losses on PASCAL VOC and MS COCO benchmarks. Results are obtained on the test set of PASCAL VOC 2007 and the val set of MS COCO 2017. mAP denotes mAP mA mAP denotes the mean AP over AP7s, su, ... AP "rela. improv." " stands for the relative improvement. = 3 is used for all a-IoU losses in all experiments.
Method
PASCAL VOC
COCO
AF5a
AP75
APs5
AP55
mAP
MAP75:05
AF50
AP75
AFs5
AP35
MAP
mAP15
Cut
78.81
58.04
35.07
2.34
52.74
312.45 45
55.51
38.59
23.58
2.07
36.
21.82
78.62
53.61
55.25
25.85
37 01
YOLOOUS
rohs improv.
-0.24%
1.27%
8.81%
55.56%
1.65%
6.21%
-0.47%
2.85%
9.63%
61.84%
1.98%
8.43%
CORU 78.19
57.77
34.89
2.36
52.30
32-17
55.67
39.01
23. 56
2. 2.03
36.36
21.95
E DiJU
78.33
50.2 24
38.46
3. 3.50
63.76
34. .6
56.84
30 499
25.49
3. 50
36.74
23 :3.34
rela. improv.
0.18%
2.54%
10.23%
48.31% %
2.79%
7.72%
0.31%
1.23%
8.19%
62.56%
1.05%
6.32%
85.24
70.08
53. 08
10.88
63.95
46.78
67. 36
52.15
38, 22
9.31
48 4..42
34. 42
CudUU
84.83
13:74
64.25
34.72
YOLOv5x
Jela. improv.
+0.48%
1.26%
26.29%
%
2.73%
0.53%
0.88%
1.05%
4.83%
0.52%
0.87%
CDLU 85.04
53. 71.05
64.21
47.30
67.54
36.
8.
48.36
94.16
Ca-UlsU
84.90
71.34
54. 23
13. 85
64.49
18. 4.0
67.42
52. 65
39, 2.8
10. 29
18. 81
3.5 422
rela. improv.
-0.16%
0.41%
0.97%
24.66%
0.44%
2.32%
-0.18%
1.19%
3,31% %
19.93%
0.89%
3.68%
CSUU
76.50
53.85 855
20. 5-4
1.62
49.78
28.82 .82
69.38
41.67
26.13
3.52
39.23
24.37
76.22
32. 35.03
2, 228
51.122
31.00
69.61
42.65
28.67
0.03
40.18
26. 44
improw.
9.34%
40.74%
2.69%
0.39%
9.34%
44.60%
2.42%
Cmet
76.26
54.09
29.
1.58
49.91
28. 68
59.
41.62
26,
3.54
39. 25
24.48
Ca-DisU
76.44
2.
50.96
59.38
rola. improv.
0.24%
1.48%
7.70%
56.41%
2.10%
6.69%
0.17%
1.73%
8.20%
51.41%
1.76%
6.41%
14000
Number of prrduccced bosus
0.6
0.5
Dtot)
0.5
0.5
Zoom In View
0.4
0.25
Zoom Mew
3000
5300
25
0.5
150
250
3000
1200
3.0
0.5
Epochs
Epochs
Figure 2: IoU distributions between predicted bboxes and their ground truth after NMS.
Figure 3: Validation mAPs (mAP across 300 training epochs.
losses are able to localize objects more accurately than the baselines with more true positives and fewer false positives.
We further analyze the bbox regression accuracy by showing the IoU distributions between the predicted bboxes and their ground truth for YOLOv5s trained using different losses on PASCAL VOC. After NMS with the IoU threshold being 0.5, we visualize the number of positively predicted bboxes under different IoU thresholds from 0.5 to 0.9 in Figure 2 showing that a-IoU losses detect more positive objects than baseline losses across all IoU thresholds. Particularly, a-IoU losses detect approximately 1% more positive objects than the baselines when IoU 0.5, and 11% more high IoU objects when ToU 2 0.9. This demonstrates that a-loU boosts both the precisions and recalls of detectors. a-IoU is extremely advantageous in pushing low IoU objects to high IoU objects by up-weighting their loss, thus outperforming baseline losses significantly at the high accuracy level and contributing to the improvement of the final detection performance (Table
Moreover, J that l- losses are able to boost the late training stage (e.g., after 200 epochs) through up-weighting the gradient of high IoU objects, while almost having no negative impact on the carly training stage (c.g., the first 100 epochs). When a > 1, the relative gradient weight is 0 < wV, < 1 for 0- < IoU < a T while 1 < wV, I a for T S ToU < 1, as analyzed in Property& and illustrated in Figure [ (right). This property helps tune down the gradients of low IoU objects at the early training stage, which has smoothing effect (reduces the high variance in parameter update caused by hard examples) that helps stabilize the model training when gradients are large at the early stage. On the other hand, the gradient up- weighting is well-bounded by wv. S a, which makes up gpighttg relatively safe for high IoU objects, as the original loss and gradient are small for these examples, so is the learning rate.
7

---

PATSION GUNZ
Figure 4: Example results the test set of PASCAL VOC 2007 using YOLOVv5 trained by Lio (top row) and La lou with a = 3 (bottom row). Lou performs better than Liou because it can localize objects more accurately (image 1 and 2), thus can detect more true positive objects (image 3 to 5) and fewer false positive objects (image 6 and 7).
Figure 5: Example results on the val set of MS COCO 2017 using YOLOv5s trained by LIo (top row) and La-boU with 3 (bottom row). Ca-IoU performs better than Lio because it can localize objects more accurately (image 1), thus can detect more true positive objects (image 2 to 5) and fewer false positive objects (image 4 to 7). Note that Lau detects both more true positive and fewer false positive objects in image 4 and 5 than LioU.
We also conduct an experiment to compare our a-IoU with a set of existing IoU-based losses in training a popular two-stage anchor- based model, Faster R-CNN (ResNet-50-FPNN) In Table results at the top are reproduced using the MMDetection toolbox [6] while those in the middle are reported Results at the bottom are obtained by replacing existing losses with their c-U versions (i.e., improve based top results using MMDetection). The results on MS COCO demonstrate that a- losses are quite competitive compared with existing baselines in terms of both mAP and mAP75.05- Note that the Autoloss searches both the classification loss and the localization loss, thus taking huge amount of searching time (23]. In contrast, a-IoU losses only need an easy modification of the localization loss and win the Autoloss without causing any additional computational overhead.
results in the original papers 44 41 23
4.3 Robustness to Noisy Bounding Boxes
It happens quite often that people annotate inaccurate bboxes in images/videos as the ground truth, even with computer-assisted annotation tools. However, there is little work on the robustness of localization losses to noisy bboxes, even though a number methods have been proposed for robust learning with noisy labels, anchors, and bboxes [27] 12, [0] [5] 37] (8, (25 19 20). Here, we fill this gap by conducting a set of experiments to evaluate the robustness of different localization losses to noisy bboxes. We show that a-loU is more robust to noisy bboxes they focus less on the low IoU objects, creating a suppression effect on the learning of the noisy bbox examples. Considering that open datasets like PASCAL VOC and MS COCO are carefully annotated, we synthesize a set of common noisy bboxes by perturbing normalized bboxes in the entire training set. The perturbations follow uniform noise distribution in [-nww nw] at horizontal coordinates (I and w) and [-rjh, vyh] at vertical coordinates (y and hi,, where vj is the noise rate 200] We then constrain all the noisy bboxes by the following boundary conditions:
0< 0
1 0<h<1
1 h y S h. 2w,
w,
(5)
2
8

---

Table 2:
The performance of Faster R-CNN (ResNN)-50-FPN) with 1x schedule and single scale training on MS COCO using different localization losses. Results are obtained on the val set MS COCO 2017. mAP denotes mAP mAP75:99 denotes the mean AP over AP7s, APs0; ... AP95- s AP APm- m. and AP; denote the AP for small, medium, and large objects, respectively. marks the reproduced results from the MMDetection toolbox [6], while marks the results in the original papers. "_" represents the missing results papers. a is used for all a-IoU losses in all experiments.
The top two best results in every column are boldfaced.
Loss
APso
AP75
APso
APss
APua
APps
MAP
mAP mAP76:95)
AP,
APm.
APt
tE2
58.13
40.45
33.56
23.39
11.09
1.21
37.37
21.95 95
21.20
10.96
18.
58.12
41.23
34.03
24.43
12.42
1.61
37.88
22.74
21.61
41.63
40.11
Casu
68. 18
41.00
33.02
24.13
11.97
1.51
37.62
22
21.49
41.07
48.90
Caaw
58.05
40.37
33.54
23.85
11.10
1.19
37.43
22.05
21.37
41.00
48.17
40.79
37.93
21.38
40.82
50.14
Letoo
41.11
21.45
41.06
50.21
41.11
38.09
21.66
41.18
50.32
41.96
38.65
21.32
41.83
$1.51
Cood Bus
59.10
42.40
38.90
21.20
41.10
50.20
"Autoloss
68.60
41.80
38.50
22.00
42.20
50.20
Codat:
58.81
41.94
31.81
25.36
13.27
1.81
38. 96
23.44 1/1
22.14
12.11
50.36
Co-
59.01
42:00
35.13
25.
13.09
2.03
399.55 18
23,46
22.05
42.19
50.08
Cu DIA
59.27
42.18
36.25
25.47
13.32
1.95
39.43
23,65
22.10
42.10
50.43
59.09
41.92
35.01
25.08
13.04
1.98
39.25
23.41
21.94
41.88
50.01
Table 3: The performance of YOLOv5s trained using different localization losses on simulated noisy trainval sets of PASCAL VOC 2007+2012 under noise rates 17 0.1, 0.2, and 0.3. Results are obtained on the clean test set of PASCAL VOC 2007. mAP denotes mAP mAP75:95 denotes the mean AP over AP7s, APso, ... APps "rela. improv." stands for the relative improvement. 3 is used for all IoU losses all experiments.
Noise
Loss
APsa
APss
APso
APes
APya
APrs
APso
APas
APso
AP9s
mAP
mAP7a:95
74.48
71.37
G8 US
6d.29
56.55
47.12
33.06
17.53
4.16
0.26
43.61
20.43
74.67
71.94
68.73
64.27
57.75
48.50
36.85
21.25
6.30
0.28
45.06
22.64
impoov.
0.26%
0.95%
1.55%
2.12%
2.93%
11.55%
51.44%
7.69%
10.85%
Ensai
74.09
71.46
67.8
63.00
56.
46.71
32.67
17.50
4.43
0.23
43.42 43.42
20.31
Ca-tnel
74.38
68. 18
35.90
0.33
23.39
rela improv.
0.39%
0.69%
0.32%
0.68%
1.78%
3.77%
9.89%
19.37%
43.79%
43.48%
2.97%
10.26%
Cau
67.82
63.93
38.22
50.
39.31
26.33
13.51
4.55
0.66
0.05
32.45
9.02
68.20
64:21
58.77
51.68
40.66
29.20
16.
6.06
1.31
0.10
33.62 63.62
10.56
0.2
rela. improv.
0,56%
0.44%
0.94%
2.95%
3.43%
10.90%
19.25%
33.19%
98.48%
100%
3.61%
17.03%
Cmea
67.39
62.94
57. 229
44. 22
313
27.
13.78
4.522
0.B8
0.02
32, 24
9.23
68,26
61.449
59.59
41.19
15.77
5.84
1.25
0.21
33.77
10.44
rela. improv.
1.29%
4.01%
5.56%
7.34%
14.44%
83.82%
13.14%
Cut
56.54
67
80
1.43
0.04
3.54
Ca-lel)
58.59
51.58
43.
32. 93
22. 27
12.52
5 1..11
2.18
0. 73
0.12
23. 00
4.29
0.3
rela, improv.
3,63%
3.80%
6.29 %
6.92%
11.41%
12.49%
22.87 %
51.05%
135%
200%
6.78%
20.99%
CDRU
56.84
49.82
41 41.50
32.06
20. 80
11.22
4.85
1.51
0.46
0.07
21.91
3.62
Cxx impoov
58.45
61.94
43.9
33.78
22.57
12.80
6.34
2.42
0.65
0.16
23.31
4.49
283%
5.78%
5.36%
30.99%
60.26%
41.30%
129%
24.09%
We test 17 0.1, 0.2, 0.3 our experiments, with the average IoU between the noisy bboxes and their clean versions dropping to 0.833, 0.710, and 0.613, respectively. Examples of the synthesized noisy bboxes can be found in Appendix B.44 As shown Table 3 a-IoU improves the baseline losses (i.e., Lio and CDIOU) considerably in these noisy scenarios. We gain increasing relative improvements from APso APss, which accumulate to more significant improvement mAP7s:95. Note that a-IoU losses also outperform the baselines at AP30 across all noisy scenarios, which is not always the case when bboxes are clean (Table []. Furthermore, a-loU losses are noticeably more robust against more severe noises. For instance, the relative improvement of La.DleU over LI increases from 2.97% 26% to 6.39% according to mAP/mAP AP7//P99 P75:95 when the noise rate v) rises from 0.1 to 0.3. These results confirm the advantage of a-IoU losses in noisy bbox scenarios.

---

mAP35
MAP
25
15
0
10
a-loU w/o
a-IoU 0.1 noise
a-DioU w/o noise
a--iou
a-loU w/ 0.2 noise
a-lou w/ 0.3 noise
a-DioU w/ 0.2 noise
D-- w/ 0.3 noise
Figure 6: The performance of YOLOv5 models trained using a-loU with different values and evaluated on the clean test set of PASCAL VOC 2007. Black dashed lines denote baselines (i.e., the family of a-IoU with 1) while red dashed lines denote the family of a-loU with 3.
4.4 Sensitivity to power parameter
a
Here, we evaluate the performance of a-loU with varying values (a E [0.5, 5)) via set of experiments with Lo- and -D-D-- The results are shown in Figure YOLOvšs on PASCAL VÓC in both clean and various noisy bbox scenarios. It evident that a-IoU losses with a E [2, perform competitively well across all scenarios, with a = 3 performing the best in most cases. When 3, a-loU losses tend perform worse low APs than the baselines (i.e., a-IoU with a 1), although the performance at high APs gains more improvement. We also test an extreme case with 10. in which the performance drops by 5.61% 21tp//1..99//.....22. 239%%/218/ on average compared with a = 3 under noise rates 0/0.1/0.22 1/..2/0..3 respectively. More specifically, it becomes worse than the baselines according to either mAP or mAP7s:95- This indicates that a proper choice of is crucial for a-IoU losses. Our recommendation is to tune E [2, 3] for most applications directly use a when tuning is too expensive. Note that E [3, 4] may be a better choice when high levels of bbox regression accuracy is desired, e.g., mAP is the preferred performance metric. It a better choice for certain applications, although a-loU losses with a < perform consistently worse than the baselines in our experiments.
possible that a <
5 Conclusions
In this paper, we proposed unified formula a-IoU to generalize existing loU-based losses to new family of power loU losses. By modulating the power parameter a, a-loU offers the flexibility to achieve different levels of bbox regression accuracy when training object detector. We analyzed the order preservingness and the loss/gradient reweighting properties of a-IoU, and showed that a-loU improve bbox regression accuracy through up- spiieight the loss and gradient of high IoU objects. Experiments with multiple detection models and benchmark datasets demonstrated that a-loU losses consistently outperform existing loU-based losses, especially at the high Average Precisions (APs). a-IoU has the potential to be widely applied in real-world object detection applications as improves existing loU-based losses, 2) it benefits light models, is extremely advantageous on small datasets, and 4) it is more robust to noisy bboxes. For future work, we will explore new generalization formulas for other netric-derived loss functions such Dice, Hausdorff distance. and Chamfer distance losses.
Societal Impacts
The proposed loss functions can help train siighperffrmance object detectors for impactful applica- tions such as self-driving, face recognition and video surveillance. While not our initial intention, these models could potentially be manipulated by adversaries unauthorized users for malicious purposes. This could compromise the safety or privacy of certain individuals. We believe striet regulations should be established to prevent such illegitimate exploitations.
10

---

References
[1] A. Bochkovskiy, C.-Y. Wang, and H.-Y. M. Liao. Yolov4: Optimal speed and accuracy of object detection. arXiv preprint 200v 2020.
[2] G. E. Box and D. R. Cox. An analysis of transformations. Journal of the Royal Staristical Society: Series (Methodological), 26(2):111244, 1964.
[3] Z. Cai and N. Vasconcelos. Cascade r-cnn: Delving into high quality object detection. In CVPR, pages 6154-6162, 2018.
[4] N. Carion, F. Massa, G. Synnaeve, N. Usunier, A. Kirillov, and S. Zagoruyko. End-to-end object detection with transformers. In ECCV, pages 213-229. Springer, 2020.
[5] K. Chen, J. Pang, J. Wang, Y. Xiong, X. Li, S. Sun, W. Feng, Z. Liu, J. Shi, W. Quyang, et al. Hybrid task cascade for instance segmentation. In CVPR, pages 4974 2019.
[6] K. Chen, Wang, J. Pang. Y. Cao, Y. Xiong, X. Li, S. Sun, W. Feng, Z. Liu, J. Xu, al. Mmdetection: Open mmlab detection toolbox and benchmark. arXiv preprint 17Xiv:9906.07155, 2019.
[7] Z. Dong, G. Li, Y. Liao, F. Wang, P. Ren, and C. Qian. Centripetalnet: Pursuing high- high-quality keypoint pairs for object detection. In CVPR, pages 00519-10528, 2020.
[8] K. Duan, S. Bai, L. Xie, H. Qi, Q. Huang. and Q Tian. Centernet: Keypoint triplets for object 2019.
detection. ICCV, pages
[9] M. Everingham, L.. Van Gool, C. K. Williams, Winn, and Zisserman. The pascal visual object classes (voc) challenge. IJCV, 88(2):303-338, 2010.
[10] J. Gao, J. Wang, S. Dai, L.-J. Li, and R. Nevatia. Note-renn: Noise tolerant ensemble renn for semi-supervised object detection. In ICCV, pages 9508-9517, 2019.
[11] R. Girshick. Fast r-cnn. In ICCV, pages 1440-1448, 2015.
[12] B. Han, Q. Yao, X. Yu, G. Niu, M. Xu, W. Hu, W. Tsang, and M. Sugiyama. Co-tcaching: Robust training of deep neural networks with extremely noisy labels. In NeurlPS, 2018.
[13] J. He, S. Erfani, S. Wijewickrema, S. O'Leary, and K. Ramamohanarao. Learning non-umique segmentation with reward penalty dice loss. IJCNN, pages 1-8. IEEE, 2020.
[14] K. He, G. Gkioxari, P. Dollár, and R. Girshick. Mask r-cnn. In ICCV, pages 2961 2961-2969, 2017.
[15] M. Khodahandeh, A. Vahdat, M. Ranjbar, and W. G. Macready. A robust learning approach to domain adaptive object detection. In ICCV, pages 480-490, 2019.
[16] H. Law and Deng. Cornernet: Detecting objects as paired keypoints. In ECCV, pages 734-750, 2018.
[17] H. Li, T. Fu, J. Dai, H. Li, G. Huang, and X. Zhu. Autoloss-zero: Searching loss functions from scratch for generic tasks, arXiv preprint arXiv:2103. 14026, 2021.
[18] H. Li, C. Tao, X. Zhu, X. Wang, G. Huang, and J. Dai. Auto seg-loss: Searching metric surrogates for semantic segmentation. arXiv preprint arXiv:2010.079.00 2020.
[19] H. Li, Z. Wu, C. Zhu, C. Xiong, R. Socher, and L. S. Davis. Learning from noisy anchors for one-stage object detection. In CVPR, pages (0588-10597) 2020.
[20] Li, C. Xiong, Socher, and S. Hoi. Towards noise-resistant object detection with noisy annotations. arXiv preprint arXiv:2003.01288 2020.
[21] T.-Y. Lin, P. Goyal, R. Girshick, K. He, and P. Dollár. Focal loss for dense object detection. In ICCV, pages 2980-2988, 2017.
[22] T.-Y. Lin, M. Maire, Belongie, J. Hays, P. Perona, D. Ramanan, Dollár, and C. L. Zitnick. Microsoft coco: Common objects in context. In ECCV, pages 740-755. Springer, 2014.

---

[23] P. Liu, G. Zhang. B. Wang. H. Xu, X. Liang, Y. Jiang, and Z. Li. Loss function discovery for object detection via convergence convergence-simulateee simulation driven search. arXiv preprint arXiv:2102.04700, 2021.
[24] W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, and A. C. Berg. Ssd: Single shot multibox detector. In ECCV, pages 21-37. Springer, 2016.
[25] X. Ma, H. Huang, Y. Wang, S. Romano, S. Erfani, and J. Bailey. Normalized loss functions for deep learning with noisy labels. In ICML, pages 6543-6553, PMLR, 2020.
[26] X. Ma, Y. Wang. M. E. Houle, S. Zhou, S. Erfani, S. Xia, S. Wijewickrema, and J. Bailey. Dimensionality-drinn driven learning with noisy labels. In ICML, pages 3355-3364, PMLR, 2018.
[27] N. Natarajan, S. Dhillon, P. K. Ravikumar, and A. Tewari. Learning with noisy labels. NeurlPS, 26: 196-1204, 2013.
[28] M. A. Rahman and Y. Wang. Optimizing intersection-over-uniooo in deep neural networks for image segmentation. In ISVC, pages 234-244. Springer, 2016.
[29] J. Redmon and A. Farhadi. Yolo9000: better, faster, stronger. In CVPR, pages 7263-7271, 2017.
[30] J. Redmon and A. Farhadi. Yolov3: An incremental improvement. arXiv preprint 10X4::8804002765 2018.
[31] S. Ren, K. He, R. Girshick, and Sun. Faster r-cnn: Towards time object detection with region proposal networks. arXiv preprint arXiv:1506.01497, 2015.
[32] H. Rezatofighi, N. Tsoi, J. Gwak, A. Sadeghian, Reid, and Savarese. Generalized intersec- tion over union: A metric and a loss for bounding box regression. In CVPR, pages 658 388-666, 2019.
[33] G. Song, Y. Liu, and X. Wang. Revisiting the sibling head in object detector. In CVPR, pages 11563-11572, 2020.
[34] Z. Tian, C. Shen, H. Chen, and T. He. Fcos: Fully convolutional one-stage object detection. In ICCV, pages 9627-9636, 2019.
[35] L. Tychsen-Smith and L. Petersson. Improving object localization with fitness nms and bounded iou loss. In CVPR, pages 6877-6885, 2018.
[36] K. Wang and L. Zhang. Single-shot two-pronged detector with rectified iou loss. In Proceedings of the 28th ACM International Conference on Multimedia, pages 1311-1319, 2020.
[37] Y. Wang, X. Ma, Z. Chen, Y. Luo, J. Yi, and J. Bailey. Symmetric cross entropy for robust learning with noisy labels. In ICCV, pages 322-330, 2019.
[38] Z. Wang, L. Liu, and D. Tao. Deep streaming label learning. In ICML, pages 9963-9972. PMLR, 2020.
[39] Yu, Jiang, Z. Wang. Z. Cao, and Huang. Unitbox: An advanced object detection network. In Proceedings of the 24th ACM international conference on Multimedia, pages 516-520, 2016.
[40] S. Zhang, C. Chi, Y. Yao, Z. Lei, and S. Z. Li. Bridging the gap between anchor-based and anchor-free detection via adaptive training sample selection. In CVPR, pages
2020.
[41] Y.FF Zhang, W. Ren, Z. Zhang, Z. Jia, L. Wang, and T. Tan. Focal and efficient iou loss for accurate bounding box regression. arXiv preprint 1xXi::2001.0815. 2021.
[42] Z. Zhang and M. R. Sabuncu. Generalized cross entropy loss for training deep neural networks with noisy labels. In NeurlPS, 2018.
[43] 7. Zheng, P. Wang, W. Liu, Li, R. Ye, and D. Ren. Distance-iou loss: Faster and better learning for bounding box regression. In AAAI, pages 12993-13000 13000, 2020.
12

---

[44] X. Zhou, D. Wang, and P. Krähenbühhl. Objects as points. arXiv preprint, arXiv: 190.0000 2019.
[45] X. Zhou, J. Zhuo, and Krahenbuhl. Bottom-up object detection by grouping extreme and center points. In CVPR, pages 830-859, 2019.
[46] X. Zhu, W. Su, L. Lu, B. Li, X. Wang, and J. Dai. Deformable detr: Deformable transformers for end-to-end object detection, arXiv preprint arX01:20100001199 2020.
13


Reviews:
[{"review_id": "r2HZvVbc_rM", "reviewer": "Reviewer_vuNT", "summary": "In this paper, the authors propose a simple but effective IoU loss. By adding a power factor on the IoU loss, the network focus more on proposals with high IoUs with ground truth. Experiments of various object detection baselines on two datasets validate the effectiveness of this loss function.\n", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "I have some concerns about applying the proposed transformation on other IoU loss variants. Besides IoU loss, the authors also add the power factor on other IoU loss functions. However, I find some inconsistencies on these loss functions. The IoU is a metric which is better when it's higher. But other metrics, such as the center distance in DIoU and aspect ratio difference, are better when they are lower. The power parameter then have different meanings on these metrics. By setting alpha>1, better bounding boxes have higher weight on IoU, but will have lower weight on other metrics. This property seems to be conflict with the authors' motivation.\nIf the authors could fix this issue, I am satisfied with the approach.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "See above.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "0.5"}, "extra_text": {}}, {"review_id": "l0b-vxnRK_Z", "reviewer": "Reviewer_LYiS", "summary": "This work proposed a new IoU based loss function-\\alpha-IoU for object detection. The proposed \\alpha-IoU can be integrated with existing IoU-based losses, e.g. GIoU, DIoU, and BIoU, and could further improve the performance. The author studied the property of the proposed method and evaluate the proposed method on several standard benchmarks.", "questions": "", "limitations": "", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Pros: The writing is good and the paper is well organized. The idea of adding the power term into the IoU loss is new for me. And extensive experiments on standard benchmarks are conducted. \n\nCons: \n- While adding the power term to the IoU loss is new, but its motivation is not clear and straightforward. In addition, the improvements compared with the baseline methods are not significant, especially in MS-COCO dataset, and the performance gain has quickly shrunk on the stronger detectors. As shown in Table 1. \n\n- As L285-L286 stated, \\alpha < 1 consistently performs worse than the baseline model. This indicates that the proposed method mainly increasing the importance of the well-localized samples. Therefore, there are some competitor methods are not compared, such as <Dynamic R-CNN>, <Focal and Efficient IOU Loss> and <Revisiting the Loss Weight Adjustment in Object Detection\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "No ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "1.5"}, "extra_text": {}}, {"review_id": "TODtvXqzhlh", "reviewer": "Reviewer_u1Yw", "summary": "Following the success of IoU-based regression losses for bounding box localization, this paper proposes a family of power IoU losses, where the IoU-related terms in these losses are regularized by a power parameter \\alpha. This idea is a direct translation of what is used in Generalized cross-entropy loss [37] and Focal loss [19] for IoU-based regression losses. By applying this new loss variant on several recent object detectors are claimed to improve their detection performance on the popular detection dataset and using the mAP and AP metrics.", "questions": "", "limitations": "", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper's idea is well clear and it is easy to follow. I have few major and critical concerns about the contribution's originality and significance, lack of theoretical proof for the claims and experimental fairness: \n\n1- Originality: This idea can be seen as a direct translation of what is used in Generalized cross-entropy loss [37] and Focal loss [19] for IoU-based regression losses and I don't see a new existing research idea \n\n2- lack of theoretical proof and false claims: First of all, this \\alpha modification of IoU and GIoU is claimed to hold the metric property of IoU and GIoU losses, (The other IoU loss variants, e.g. CIoU, DIoU etc, may not be even metric losses. ). I don't see a proper mathematical proof fulfilling this claim in the main text or supp. materials. The authors have a sentence in line 155 page 4 mentioned that the power transformation preserves key metric property and cite the paper [28]. I didn't find such proof in the GIoU paper for the power transformation. Property 1 might be helpful for proving this lemma. But even property is only a naive claim and no mathematical proof is available.\n\nI am also not fully convinced that if we optimize a loss (\\alpha-IoU) that is not directly correlated with IoU as a metric, it would be useful compared to optimizing the metric. why it makes sense to use a loss it has a high gradient in high IoU and a very low gradient in low IoU. More insightful theoretical analysis is required to support the usefulness of the proposed losses\n\n3- Experimental fairness: First of all, I couldn't find the detailed implementation details for all the experiments in Tables1-3 and how hyperparameters are set for the baselines. I wonder this improvement is rather a side effect of dynamic weightings against classification losses over iterations rather than the effect of the regression loss itself. Obviously, the Intersection over Union between prediction and GT boxes increases over the training iterations makes the regression loss bigger in the case of \\alpha-IoU. Detection is a multi-task problem and regularization between the regression and classification losses matters considering their effect in AP and mAP metric.   More detailed experiments are required to substantiate the source of the improvement ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Better localization of objects might be conducive to reduce the risk of collision in autonomous driving applications.", "review_text": "", "extra_scores": {"ethical_concerns": "N/A", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The submission considers the use of a power term in the IoU for bounding box regression.  By exponentiating the IoU in the construction of the loss, the gradients will be higher for high/low IoU depending on whether the parameter is greater or less than one.  The reviewers were unanimous that the paper is not suitable for publication at NeurIPS at this time, and the opinion remained during the post rebuttal discussion.  Among the main concerns about this paper were: that the reported benefits of the method decrease with stronger detectors and are small compared to e.g. Dynamic RCNN, and that the idea is quite similar to generalized cross-entropy loss and Focal loss limiting novelty.
Decision: Accept (Poster)