$\alpha$-IoU: A Family of Power Intersection over Union Losses for Bounding Box Regression
Authors: Jiabo He
Sarah Monazam Erfani
Xingjun Ma
James Bailey
Ying Chi
Xian-Sheng Hua
Venue: NeurIPS 2021 Poster (2021)
Keywords: bounding box regression
localization loss
object detection
intersection over union

Abstract: Bounding box (bbox) regression is a fundamental task in computer vision. So far, the most commonly used loss functions for bbox regression are the Intersection over Union (IoU) loss and its variants. In this paper, we generalize existing IoU-based losses to a new family of power IoU losses that have a power IoU term and an additional power regularization term with a single power parameter $\alpha$. We call this new family of losses the $\alpha$-IoU losses and analyze properties such as order preservingness and loss/gradient reweighting. Experiments on multiple object detection benchmarks and models demonstrate that $\alpha$-IoU losses, 1) can surpass existing IoU-based losses by a noticeable performance margin; 2) offer detectors more flexibility in achieving different levels of bbox regression accuracy by modulating $\alpha$; and 3) are more robust to small datasets and noisy bboxes.

Alpha-loU: A Family of Power Intersection over Union Losses for Bounding Box Regression
Jiabo Hel. Sarah Erfani', Xingjun Ma² James Bailey', Ying Chi Xian-Sheng Hua
'School of Computing and Information Systems, The University of Melbourne School of Computer Science, Fudan University
DAMO Academy, Alibaba Group (jiaboh@student. sarah. verah.erfanil baileyj@/unimelb edu. danxjma@gmaii com, (xinyi.cy cy, xiansheng. hxs) Saaeeaaaaaaaaaaaaaeaaeaaaaaaaaaa wwwaaannaaaaaaaaaaaaaaaaaaaaaaaa com
Abstract
Bounding box (bbox) regression is a fundamental task in computer vision. So far, the most commonly used loss functions for bbox regression the Intersection over Union (loU) loss and its variants. In this paper, we generalize existing loU- family of power IoU losses that have power IoU term and an additional power regularization term with a single power parameter a. We call this new family of losses the a-IoU losses and analyze properties such as order preservingness and loss/gradient reweighting. Experiments on multiple object detection benchmarks and models demonstrate that a-IoU losses, 1) surpass existing IoU-basee losses by noticeable performance margin; 2) offer detectors more flexibility in achieving different levels of bbox regression accuracy by modulating o; and 3) are more robust to small datasets and noisy bboxes.
based losses
Introduction
Bounding box (bbox) regression localizes an object in an image/video by predicting bbox for the object, which is fundamental to object detection, localization, and tracking. For example, the most advanced object detectors often consist of bbox regression branch and classification branch with the bbox regression branch generating bboxes to localize objects for classification. In this work, we explore more effective loss functions for bbox regression in the context of object detection.
Whilst early works object detection use -nom losses [ for bbox regression, recent works directly adopt the localization performance metric, i.e., Intersection over Union (IoU), the local- ization loss (28 39). Compared with -n-or losses, the IoU loss invariant to bbox scales, thus helping train better detectors. However, the IoU loss suffers from the gradient vanishing problem when the predicted bboxes are not overlapping with the ground truth, which tends to slow down convergence and result in inaccurate detectors. This has motivated the design of several improved loU-based losses including Generalized loU (GloU), Distance-loU (DIoU) and Complete loU (CloU). GloU introduces a penalty term into the IoU loss to alleviate the gradient vanishing problem [32) while DIoU and CloU consider the central point distance and aspect ratio between predicted bboxes and their ground truth in penalty terms [43]
this paper, we present new family of IoU losses obtained by applying power transformations to existing loU-based losses. We first apply the Box-Cox transformation 2 to the loU loss Lio 1- IoU and generalize it to power IoU loss: La-toU = (1 - IoU )/a, a > 0, denoted as a-loU. We further simplify a-IoU to La-toU = 1 I for 0 and extend to more general form
35th Conference Neural Information Processing Systems (NeurIPS 2021).
'This research was done when Jiabo He interned with the DAMO Academy, Alibaba Group. t Cooresponding authors.

---

with an additional power regularization term (see equation (3)) This allows us to generalize existing loU-based losses, including GloU, DIoU, and CloU, to a new family of power IoU losses (see equation (4) for more accurate bbox regression as well as object detection.
We show that, relative to LioU, loU with a > 1 up-weights both the loss and gradient of high loU objects, leading to improved bbox regression accuracy. When 0 < a < 1, it down-weights high IoU objects which we find hurts regression accuracy. The power parameter a can serve as knob to adapt a-loU losses to meeting different levels of bbox regression accuracy (precision measured under different IoU thresholds). with a > 1 for high regression aceuracy (i.e., high IoU thresholds) by focusing more on those high IoU objects. We also empirically show that a is not overly sensitive to different models or datasets, with a = 3 performing consistently well in most cases. The family of a-loU losses can be easily applied for improving state-of state-of-the-art detectors under both clean and noisy bbox settings without introducing additional parameters to these models (making any modifications to training algorithms), nor increasing their training/inference time.
In summary, our main contributions are follows:
We propose new family of power IoU losses called I-U for accurate bbox regression and object detection. a-IoU presents a unified power generalization of existing loU-based losses.
We analyze a set of properties of a-IoU, including order preservingness and loss/gradient reweight- ing, to show that a proper choice of a (i.e., a > 1) can help improve bbox regression accuracy by adaptively up- up-weithting the loss and gradient of high IoU objects.
We empirically show, on multiple benchmark object detection datasets and models, that a-IoU losses can consistently outperform existing loU-based losses and provide more robustness for small datasets and noisy bboxes.
2 Related Work
Object Detection Models. There exist two mainstream types of detection models: anchor-based and anchor-free detectors. Anchor-basee detectors can be further divided into two-stage and one- stage models. Two-stage unchor-based detectors (c.g., R-CNN series [I] 31 14 3], HTC (5). and TSD 33) are firstly proposed in object detection tasks, which are composed of region proposal networks (RPNs) and classifiers. RPNs generate large number of foreground and background region proposals, followed by networks to classify objects in the proposals. Towards real-time object detection, one-stage unchor-bases detectors (e.g., YOLO series [29 30 I. RetinaNet [21]. and SSD
(24 are developed to predict bboxes and categories at the same time, thus no longer need RPNs. Anchor boxes with prior scales and aspect ratios should be defined before training anchor-based detectors. Techniques have been proposed to mitigate the sensitivity of these models to hand-picked anchor boxes, for example, attention- fusion networks 33 and clustering algorithms [30). These techniques learn prior anchors from the training set for every sliding window or grid cell.
Recently, anchor-free detectors such as CornerNet [16). CenterNet, (8] ExtremeNet [45), and CentripetalNet [7]. have also been proposed to get rid of anchor priors. These models first predict locations of keypoints (corners, centroids, or extreme points), then group them into the same bboxes if they are geometrically aligned. There also exist other models that generate pixel-wise results. For example, CenterNet, estimates pixel-level categories of objects along with their sizes and offsets 44). FCOS generates pixel-wise classification, centerness, and bbox (top, down, left, right) results using multi-head CNNs (34), followed by the Adaptive Training Sample Selection (ATSS) 40 as an improvement on automatically selecting positive and negative samples. In addition, transformers (c.g., DETR series 4 (66) have also been developed for object detection without anchor generation par with the above CNN-based
or non-maximum suppression (NMS), achieving the performance detectors. In this work, we propose a new family of generalized IoU losses to improve the performance of these detectors without any architectural modifications, which is orthogonal to the above research.
Bounding Box Regression Losses. anchor-based detectors regress offsets between ground-trute bboxes and their closest anchors, while anchor-free detectors predict keypoints of objects with some frameworks also generating the sizes of the bboxes. The predicted offsets or keypoints (w/ or w/o bbox sizes) are then mapped back to the pixel space for generating the bboxes. Localization losses usually compare the generated bboxes with their ground truth. Early works adopt -o-mo losses (II] for bbox regression, which have been found sensitive to varying bbox scales. Recent works replace

---

(X Y). the task space to the annotation space f X + Y. In object detection, each y; = (ct,k) e B.,,) yo, k= where m. them with the IoU loss and its variants such as BloU, GloU, DIoU and CloU for bbox regression, as IoU the metric for localization and it is scale-invariant 28 39 The Bounded IoU (BIoU) loss maximizes the loU overlap between the region of interest (Rol) and the ground truth based on set of ToU upper bounds [35] GloU is proposed address the problem of gradient vanishing on non-overlapping examples, which are examples having non-overlapping predicted bboxes with the ground truth (IoU is zero) [32]. DIoU and CloU (43) losses further consider the overlapping area, central point distance, and aspect ratio in IoU and the regularization terms. These regularization terms can help improve the convergence speed as well as the final detection performance. There are also losses designed to focus more on high IoU objects, for example, the Rectified IoU (RIoU) loss [36], and the Focal and Efficient loU (Focal-EloU) loss [41]. These loss functions increase gradients of those examples that are in high bbox regression accuracy. However, RIoU and Focal- EooU are neither concise nor generalized compared with other loU-based losses. In this paper, we apply a power transformation to generalize the above vanilla IoU loss and regularized loU-based losses for both their IoU and regularization terms. The new family of generalized losses improve bbox regression accuracy by adaptively reweighting the loss and gradient of high and low IoU objects.
There are also works on AutoML-based loss function search for computer vision tasks 23 [8 17 Despite their advantage in saving human efforts, these methods are very expensive in searching qualified loss functions (e.g., days of searching time on multiple GPUs), and probably with limited performance improvement based on existing losses [23). We will empirically compare with one of these losses in our experiments.
3 a-IoU Losses for Bounding Box Regression
3.1 Preliminaries
We study the problem of bbox regression in object detection. Let X E RRd be the input space and Y RR, be the annotation space, with d, and dy denoting the input and annotation dimensions, respectively. Given a dataset D = {( y:)} of n training examples with each (æ., y:) E learn a function f (represented by a detector network) that maps the input
is the total number of objects in æ; Ci,e is the category of the 1tth object in æ; and B, is its bbox.
The bbox regression performance is measured by the Intersection over Union (loU) metric between the predicted bbox B and the ground truth 8 IoU = / B Positive examples (both true and false positives) are determined from the set of predictions according to IoU threshold, based on which the Average Precision (AP) over all categories of objects can be calculated. E.g.. AP50 measures the AP of objects localized by bboxes with an IoU that is above the threshold 0.5. The final performance of detector is commonly evaluated by the mean Average Precision (mAP) across multiple IoU thresholds. For instance, the popular metric mAP measures the mAP of examples across the set of IoU thresholds ranging from 0.5 to 0.95 with a stride of 0.05.
3.2
I- Losses
The vanilla IoU loss is defined as Lis 1 ToU. and generalize the loU loss to an a-loU loss:
We first apply the Box-Cox transformation
1 1 Io
Ca-1oU
a a >
By modulating the parameter a in a-loU, one can derive most of the IoU terms in existing losses, e.g., log( (og((o)),, ), Io and loU2, When a - O. we obtain lima-+0 Ca-toU = loo( = LiggtoU (see the proof in Appendix(A). We recover the loU loss with a 1: LI-WUU
IoU LIoU- And C2 = ] (1- - 2 = 5 Liau, when a = 2. We can also extend the above I- formula to
39
loss functions with multiple IoÚ terms (e.g. RIoU (36 by using multiple a values.
"The Box-Cox transformation has also been applied for generalizing the Cross Entropy (CE) loss into the Generalized Cross Entropy (GCE) loss 42]. Such a generalization unifies the Mean Absolute Error (MAE) and the CE loss, increasing the robustness to noisy labels for classification tasks. However, GCE may slow down convergence and lead to degraded performance [37 25). In contrast, our a-IoU generalization is for bbox regression and can improve the localization performance both clean and noisy bbox datasets.
3

---

High loU examples
High laU examples
0.8
Gradientt
3
Loss
0.2
1
0
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
IoU
loU
.x-0.5
m=1 m=1.
o==
2=2.5
0=3
0055
6=1
#=1.5
a=2
2=5.
rr=3
Figure 1: Correlation between IoU and Lo- 1 - Io (left) and its absolute gradient IVou (right) with different a E [0.5,3], According both plots. Lo- reweights all objects adaptively and distinctively for 0 < < 1s.. a (a= marks the IoU loss).
We simplify the above a-IoU formula for > 0 and + as this case, the denominator a in equation ①1 is just positive constant in the objective. This gives us two cases of the a-loU loss for -> and -> respectively:
= { (oog([ToU) -- 0,
La-IoU
(2)
- IoUO a+ O.
Here, more interested in the 2 1. We then extend the above a-IoU loss for a + to a more general form by introducing power penaltyyfeegularization term into the formula:
O as most state-of-the-Bn IoU-based losses have an
La = TOU + pad (B, Bot),
where 0 > 0, a2 > 0, and p (B, Bst) denotes any penalty term computed based on B and Bst, This simple extension allows straightforward generalization of existing IoU-based losses their a-IoU versions. In Appendix I.2.I we empirically show that Lo- is not sensitive to a22. We thus maintain the power consistency between the IoU term and the penalty term and take a1 02 as simple choice when training the detectors.
With the above loU formula, LIuU, LGoU LDOUU, and LCIoU using the same power parameter a for the loU and penalty terms:
now generalize the commonly used IoU-based losses including
LioU IoU = La-IoU = - IoUa
IC |C\(BU (BU B
La-GioU = 1 L + CV (BUBB B
C
CGIOU = IoU +
|C|
|C|
p² bor)
Da.DIoU = LoU + 2 (2 bot)
LDIOU = ToU +
c²
ca0
LCIU = 1 ToU + (b, b
(b, bat
+ 3r Ca.
ToU +
(Bu)a.
c²
220
where C in LaNO denotes the smallest convex shape enclosing B and B and ae in LDIOU denote central points of B and Bat with p(-) being the Euclidean distance and being the diagonal length of the smallest enclosing box; and in LCioU - (arctan / arctan w) give us the family of power IoU losses for bbox regression with their original versions recovered at = 1. Note that the above a-loU generalization can be easily extended to more complex loss functions that have multiple loU or penalty terms (e.g., Clat). Next, we will analyze the properties of a-loU losses when a takes different values.
T They
3.3 Properties of a-loU Losses
Here, we focus on the vanilla a-loU formula L.-IoU I0U to analyze properties, as penalty terms may affect these properties differently. Figure[iillustrate the correlation between loU and LoU (left) and the magnitude of its gradient w.r.t. ToU, i.e., Viouaa (right). One key observation is that the IoU loss (i.c.. 1) has a linear correlation with IoU and the gradient

---

is a constant, while La- reweights objects adaptively (according to their IoU values) following different reweighting schemes with < versus a 1.
The power transformation in LoU preserves key properties of LioU as a performance metric, includ- ing non-negativity, identity of indisceribless, symmetry, and triangle inequality [32]. Furthermore, we analyze the following important properties of La-toU with detailed derivations deferred to Appendix We first let B; and B3 be two predicted bboxes by two different models M, and M, respectively, and B; and B, correspond to the same ground truth Bst with IoU (B., BBy) < (B, Bot). Then we have the first property of La-ooU
Property 1 (Order Preservingness). fa-ooU preserves the orders of both loU and Liou: (B., B < [o, B5t) = LIou(B,, Bot) > Liou(Bj, B9t) = Lastoul B9t) La-tou Bj, Bot).
The above property indicates that both La- and LioU are monotonically decreasing functions w.r.t. IOU. As Lo- preserves the order of LiaU strictly, it is guaranteed that arg min min (B, B is identical to arg max g IlU (B, Bst) and arg mingCwv [, 3. In other words, the optimal solution arg max y g/o (B, Bsy) can be obtained by minimizing either Lo--oll or LIoU. Following this, the adaptive relative loss reweighting scheme of La-ooU can be characterized by the second property:
Property 2 (Relative Loss Reweighting). Compared with LioUs La- adaptively reweights the relative loss of all objects by wc, = Lo Liot = 1+ (IoU- - // 0) == 1, and lim 1oU-- Wc. = a.
with we, (ToU' (I.(loU' =
The second property indicates that La-tou will adaptively down-weight and up- weight the relative loss of all objects according to their IoUs when 0 < < 1 and a > 1, respectively. We further note that, when a > 1, the reweighting factor wc. increases monotonically with the increase of loU (w.. grows from 1 to a') while decreasing monotonically with the increase of loU when 0 < a < 1 (we.. decays from 1 to a:). We will empirically show that the up- up-weighting scheme of La with a > 1 can help the model focus more on high IoU objects improve both the localization (i.e., predict more high IoU objects) and detection (i.e., more accurate at high APs) performance]] Similarly, we can obtain the third property of adaptive relative gradient reweighting owned by Ca-LoU as follows:
Property 3 (Relative Gradient Reweighting). Compared with LioUs Lo-toU adaptively reweights the relative gradient of all objects by wv. = Wowuaatoos V VouLl = aIooo0-11, with the turning point at IoU = =- E (0, !) when 0 < a < and IoU = T E (2. when a > 1.
When a > the above reweighting factor wv, increases monotonically with the increase of IoU, while decreasing monotonically with the increase of loU when 0 < a < 1. This relative gradient reweighting scheme is also adaptive to IoU, with the turning point from up up-weighting to down- weightiig at IoU- = aa-aa E (0, 2) when 0 < a < 1, and from down-weightig to up-
weighting at IoU = - E (2, when a > 1. The gradient reweighting scheme is bounded i.e., 0 < wv S a when a > 1, and wv, 2 when 0 < a < 1. This relative gradient reweighting scheme allows the model to learn objects with adaptive speeds (i.e.. different gradients) according to their IoUs. Theoretically, when
by wv = 1)
2. V > ViauCioul for ToU E (0.5, 1]. which accelerates the learning of all positive IoU objects at AP However, we empirically show that a-loU losses with a = 3 perform more competitively than those with a = 2 in most cases. It is probable that a-IoU losses with a = 3 further up up-weight the relative loss of objects with LoU E (0.5, 1J. although a-I losses with = 2 also beat existing basclines (see Figure 6). This property is both data-agnostic and model-agnostic, so we recommend a = 3 or a E [2, in practical use for other datasets and models.
The above loss and gradient reweighting schemes can also be inferred from Figure I with detailed proofs in Appendix A To summarize, La-toU trains better detectors than LIo for the following reasons. First, the same optimal loU can be achieved by Lo as that by LisU (Property 1). Second, Loo with a > 1 focuses more on high IoU objects by up-weighting their relative loss (Property
"The relative loss reweighting scheme of La-bo is reminiscent of the reweighting scheme of the focal loss FL( - (1 ") log(p). 7 O, which was proposed encourage the learning of low confidence foreground objects the presence of large number of high confidence backgrounds 2I]. In contrast, La-i with a > I focuses more on high IoU objects as the final performance is comprehensively measured by mAPs0.ss. with low IoU objects (here ToU < 0.5) suppressed at evaluation. Note that focal loss is usually designed for the classification branch of object detectors while our alloU is for the bbox regression branch.
5

---

2). Third, LoU with a > 1 helps detectors learn faster on high loU objects (here ToU E (aT=o, ID) through up-weighting their relative gradient (Property 3). In Appendix [A] we also provide an analysis of the absolute loss and gradient reweighting properties (Property 4 and 5), showing the additions of a-IoU to loU. Specifically, when a > 1, La-toU adds an absolute loss weight to LioU (i.e.. Wc. La-toU LioU LoU IoU > 0 for LoU E (0, 1)), which creates more space for optimization on all levels of objects (Property 4). Likewise, La-loU puts an absolute gradient aloUa-1
weight for high IoU objects (i.e., wv. |VouLa-toul - Wiwl
> 0 for IoU E (a--a, 1) such that the lcarning of high IoU objects is accelerated (Property 5). Both of the absolute and relative properties of La-IoU are adaptive to the IoU values of the objects. Such reweighting schemes will provide more flexibility in achieving different levels of bbox regression accuracies (AP measured under different IoU thresholds).
Learning Dynamics of La- Training with Ca-toU is a dynamic process and should be interpreted based on both the absolute and relative properties. With a > 1, easy examples will be learned first with increasing speed towards IoU = 1, while hard examples will be learned gradually and accelerated later on as their IoU improves. We will empirically show in Figure that up-weighting the loss and gradient of high IoU objects can boost the training at the later stage. As a comparison, we will also show that a-IoU losses with 0 < a < 1 tend to degrade the final performance in Section Reducing the loss and gradient of high IoU objects ends up with more poorly localized objects.
4.4
4 Experiments
4.1 Datasets and Training Setup
We conduct all experiments on two popular henchmarks, i.e., PASCAL VOC [9] and MS COCO 22 On the PASCAL VOC benchmark, we train all models on the trainval set 2007+2012 (containing 16, 551 images from 20 categories) and evaluate them on the test set 2007 (containing 4, 952 images) 29. On the MS COCO benchmark, we train all models on the training set 2017 (containing 118K images from 80 categories) and evaluate them on the val set 2017 (containing 5K images) [22]. We train all state-of-the-art models with the original implementation released by the authors. Specifically, we follow the original implementation' 's training protocol with default parameters and the number of training epochs with different losses [3] 32 43 4). Implementation details of all models are
given in Appendix [B.]] All experiments are run with NVIDIA V V100 GPUs. Code is available at https://gtt wtt.bb..gttuu.t.//gg com/ aom/Jaaobb99/A1pha IoU
4.2 Results and Analysis
We first validate the effectiveness of a-IoU losses in training both anchor-based and anchor-free models on the two datasets. We choose YOLOv5s (i.c., YOLOVS small) and YOLOv5x (i.c., YOLOv5 extra large) as one-stage anchor-based models, and DETR (ResNet-50) as an anchor-free model. Both a-IoU losses (i.c., Ca-toU and Ca-Diou) are generalized from existing baselines following equation (4). From Table T. we can observe that a-U losses surpass existing losses consistently across multiple models and datasets in terms of both mAP and mAP75.95. especially at the high bbox regression accuracy mAP73:93. The superiority of a-IoU losses is more pronounced at high accuracy levels, which might reach more than 60% relative improvement at AP Interestingly, a-IoU losses tend to help more of light models (e.g., YOLOv5s with 7.3M parameters and 17 GFLOPs) than heavy models (e.g., YOLOVV with 87. 7..7M parameters and 218.8 GFL.Pss.. This indicates that a- losses hold more advantage while training light models in computing-resouse -imoourrre-mmiteed scenarios, such mobile devices, autonomous vehicles, and robots.
The consistent improvements on both PASCAL VOC and MS COCO demonstrate the stability of a-IoU losses across different datasets. In addition, we also verify its robustness to extremely small training sets in Appendix B.2.2] where a-IoU losses beat existing losses at various scales, i.e., from 4K (25% trainval set of PASCAL VOC 2007+2012) to 1 118K (the entire training set of MS COCO 2017) samples. It is possible that a-U losses may not perform well if measured by a single low AP metric. For example, there may be less than 0.5% performance drop at AP5o when = 3, however, this is compensated by the significant boost at high APs. With some examples from the test set of PASCAL VOC 2007 (Figure and the val set of MS COCO 2017 (Figure we show that a-loU

---

Table 1: The performance of YOLOv5s, YOLOv5x and DETR models trained using different localization losses on PASCAL VOC and MS COCO benchmarks. Results are obtained on the test set of PASCAL VOC 2007 and the val set of MS COCO 2017. mAP denotes mAP mA mAP denotes the mean AP over AP7s, su, ... AP "rela. improv." " stands for the relative improvement. = 3 is used for all a-IoU losses in all experiments.
Method
PASCAL VOC
COCO
AF5a
AP75
APs5
AP55
mAP
MAP75:05
AF50
AP75
AFs5
AP35
MAP
mAP15
Cut
78.81
58.04
35.07
2.34
52.74
312.45 45
55.51
38.59
23.58
2.07
36.
21.82
78.62
53.61
55.25
25.85
37 01
YOLOOUS
rohs improv.
-0.24%
1.27%
8.81%
55.56%
1.65%
6.21%
-0.47%
2.85%
9.63%
61.84%
1.98%
8.43%
CORU 78.19
57.77
34.89
2.36
52.30
32-17
55.67
39.01
23. 56
2. 2.03
36.36
21.95
E DiJU
78.33
50.2 24
38.46
3. 3.50
63.76
34. .6
56.84
30 499
25.49
3. 50
36.74
23 :3.34
rela. improv.
0.18%
2.54%
10.23%
48.31% %
2.79%
7.72%
0.31%
1.23%
8.19%
62.56%
1.05%
6.32%
85.24
70.08
53. 08
10.88
63.95
46.78
67. 36
52.15
38, 22
9.311
48 4..42
34. 42
CudUU
84.83
13:74
64.25
34.72
YOLOv5x
Jela. improv.
+0.48%
1.26%
26.29%
%
2.73%
0.53%
0.88%
1.05%
4.83%
0.52%
0.87%
CDLU
85.04
53. 71.05
64.21
47.30
67.54
36.
8.
48.36
94.16
Ca-UlsU
84.90
71.34
54. 23
13. 85
64.49
18. 4.0
67.42
52. 65
39, 2.8
10. 29
18. 81
3.5 422
rela. improv.
-0.16%
0.41%
0.97%
24.66%
0.44%
2.32%
-0.18%
1.19%
3,31% %
19.93%
0.89%
3.68%
CSUU
76.50
53.85 855
20. 5-4
1.62
49.78
28.82 .82
69.38
41.67
26.13
3.52
39.23
24.37
76.22
322 35.03
2, 228
51.122
31.00
69.61
42.65
28.67
0.03
40.18
26..44 44
improw.
9.34%
40.74%
2.69%
0.39%
9.34%
44.60%
2.42%
Cmet
76.26
54.09
29.
1.58
49.91
28. 68
59.
41.62
26,
3.54
39. 25
24.48
Ca-DisU
76.44
2.
50.96
59.38
rola. improv.
0.24%
1.48%
7.70%
56.41%
2.10%
6.69%
0.17%
1.73%
8.20%
51.41%
1.76%
6.41%
14000
Number of prruuccced bosus
0.6
0.5
Dtot)
0.5
0.5
Zoom In View
0.4
0.25
Zoom Mew
3000
5300
25
0.5
150
250
3000
1200
3.0
0.5
Epochs
Epochs
Figure 2: IoU distributions between predicted bboxes and their ground truth after NMS.
Figure 3: Validation mAPs (mAP across 300 training epochs.
losses are able to localize objects more accurately than the baselines with more true positives and fewer false positives.
We further analyze the bbox regression accuracy by showing the IoU distributions between the predicted bboxes and their ground truth for YOLOv5s trained using different losses on PASCAL VOC. After NMS with the IoU threshold being 0.5, we visualize the number of positively predicted bboxes under different IoU thresholds from 0.5 to 0.9 in Figure 2 showing that a-IoU losses detect more positive objects than baseline losses across all IoU thresholds. Particularly, a-IoU losses detect approximately 1% more positive objects than the baselines when IoU 0.5, and 11% more high IoU objects when ToU 2 0.9. This demonstrates that a-loU boosts both the precisions and recalls of detectors. a-IoU is extremely advantageous in pushing low IoU objects to high IoU objects by up-weighting their loss, thus outperforming baseline losses significantly at the high accuracy level and contributing to the improvement of the final detection performance (Table
Moreover, J that l- losses are able to boost the late training stage (e.g., after 200 epochs) through up-weighting the gradient of high IoU objects, while almost having no negative impact on the carly training stage (c.g., the first 100 epochs). When a > 1, the relative gradient weight is 0 < wV, < 1 for 0- < IoU < a T while 1 < wV, I a for T S ToU < 1, as analyzed in Property& and illustrated in Figure [ (right). This property helps tune down the gradients of low IoU objects at the early training stage, which has smoothing effect (reduces the high variance in parameter update caused by hard examples) that helps stabilize the model training when gradients are large at the early stage. On the other hand, the gradient up- weighting is well-bounded by wv. S a, which makes up gpighttg relatively safe for high IoU objects, as the original loss and gradient are small for these examples, so is the learning rate.
7

---

PATSION GUNZ
Figure 4: Example results the test set of PASCAL VOC 2007 using YOLOVv5 trained by Lio (top row) and La lou with a = 3 (bottom row). Lou performs better than Liou because it can localize objects more accurately (image 1 and 2), thus can detect more true positive objects (image 3 to 5) and fewer false positive objects (image 6 and 7).
Figure 5: Example results on the val set of MS COCO 2017 using YOLOv5s trained by LIo (top row) and La-boU with 3 (bottom row). Ca-IoU performs better than Lio because it can localize objects more accurately (image 1), thus can detect more true positive objects (image 2 to 5) and fewer false positive objects (image 4 to 7). Note that Lau detects both more true positive and fewer false positive objects in image 4 and 5 than LioU.
We also conduct an experiment to compare our a-IoU with a set of existing IoU-based losses in training a popular two-stage anchor- based model, Faster R-CNN (ResNet-50-FPNN) In Table results at the top are reproduced using the MMDetection toolbox [6] while those in the middle are reported Results at the bottom are obtained by replacing existing losses with their c-U versions (i.e., improve based top results using MMDetection). The results on MS COCO demonstrate that a- losses are quite competitive compared with existing baselines in terms of both mAP and mAP75.05- Note that the Autoloss searches both the classification loss and the localization loss, thus taking huge amount of searching time (23]. In contrast, a-IoU losses only need an easy modification of the localization loss and win the Autoloss without causing any additional computational overhead.
results in the original papers 44 41 23
4.3 Robustness to Noisy Bounding Boxes
It happens quite often that people annotate inaccurate bboxes in images/videos as the ground truth, even with computer-assisted annotation tools. However, there is little work on the robustness of localization losses to noisy bboxes, even though a number methods have been proposed for robust learning with noisy labels, anchors, and bboxes [27] 12, [0] [5] 37] (8, (25 19 20). Here, we fill this gap by conducting a set of experiments to evaluate the robustness of different localization losses to noisy bboxes. We show that a-loU is more robust to noisy bboxes they focus less on the low IoU objects, creating a suppression effect on the learning of the noisy bbox examples. Considering that open datasets like PASCAL VOC and MS COCO are carefully annotated, we synthesize a set of common noisy bboxes by perturbing normalized bboxes in the entire training set. The perturbations follow uniform noise distribution in [-nww nw] at horizontal coordinates (I and w) and [-rjh, vyh] at vertical coordinates (y and hi,, where vj is the noise rate 200] We then constrain all the noisy bboxes by the following boundary conditions:
0< 0
1 0<h<1
1 h y S h. 2w,
w,
(5)
2
8

---

Table 2:
The performance of Faster R-CNN (ResNN)-50-FPN) with 1x schedule and single scale training on MS COCO using different localization losses. Results are obtained on the val set MS COCO 2017. mAP denotes mAP mAP75:99 denotes the mean AP over AP7s, APs0; ... AP95- s AP APm- m. and AP; denote the AP for small, medium, and large objects, respectively. marks the reproduced results from the MMDetection toolbox [6], while marks the results in the original papers. "_" represents the missing results papers. a is used for all a-IoU losses in all experiments.
The top two best results in every column are boldfaced.
Loss
APso
AP75
APso
APss
APua
APps
MAP
mAP mAP76:95)
AP,
APm.
APt
tE2
58.13
40.45
33.56
23.39
11.09
1.21
37.37
21.95 95
21.20
10.96
18.
58.12
41.23
34.03
24.43
12.42
1.61
37.88
22.74
21.61
41.63
40.11
Casu
68. 18
41.00
33.02
24.13
11.97
1.51
37.62
22
21.49
41.07
48.90
58.05
40.37
33.54
23.85
11.10
1.19
37.43
22.05
21.37
41.00
48.17
40.79
37.93
21.38
40.82
50.14
Letoo
41.11
21.45
41.06
50.21
41.11
38.09
21.66
41.18
50.32
41.96
38.65
21.32
41.83
$1.51
Cood Bus
59.10
42.40
38.90
21.20
41.10
50.20
"Autoloss
68.60
41.80
38.50
22.00
42.20
50.20
Codat:
58.81
41.94
31.81
25.36
13.27
1.81
38. 96
23.44 1/1
22.14
12.11
50.36
Co-
59.01
42:00
35.13
25.
13.09
2.03
399.55 18
23,46
22.05
42.19
50.08
Cu DIA
59.27
42.18
36.25
25.47
13.32
1.95
39.43
23,65
22.10
42.10
50.43
59.09
41.92
35.01
25.08
13.04
1.98
39.25
23.41
21.94
41.88
50.01
Table 3: The performance of YOLOv5s trained using different localization losses on simulated noisy trainval sets of PASCAL VOC 2007+2012 under noise rates 17 0.1, 0.2, and 0.3. Results are obtained on the clean test set of PASCAL VOC 2007. mAP denotes mAP mAP75:95 denotes the mean AP over AP7s, APso, ... APps "rela. improv." stands for the relative improvement. 3 is used for all IoU losses all experiments.
Noise
Loss
APsa
APss
APso
APes
APya
APrs
APso
APas
APso
AP9s
mAP
mAP7a:95
74.48
71.37
G8 US
6d.29
56.55
47.12
33.06
17.53
4.16
0.26
43.61
20.43
74.67
71.94
68.73
64.27
57.75
48.50
36.85
21.25
6.30
0.28
45.06
22.64
impoov.
0.26%
0.95%
1.55%
2.12%
2.93%
11.55%
51.44%
7.69%
10.85%
Ensai
74.09
71.46
67.8
63.00
56.
46.71
32.67
17.50
4.43
0.23
43.42 43.42
20.31
Ca-tnel
74.38
68. 18
35.90
0.33
23.39
rela improv.
0.39%
0.69%
0.32%
0.68%
1.78%
3.77%
9.89%
19.37%
43.79%
43.48%
2.97%
10.26%
Cau
67.82
63.93
38.22
50.
39.31
26.33
13.51
4.55
0.66
0.05
32.45
9.02
68.20
64:21
58.77
51.68
40.66
29.20
16.
6.06
1.31
0.10
33.62 63.62
10.56
0.2
rela. improv.
0,56%
0.44%
0.94%
2.95%
3.43%
10.90%
19.25%
33.19%
98.48%
100%
3.61%
17.03%
Cmea
67.39
62.94
57. 229
44. 22
313
27.
13.78
4.522
0.B8
0.02
32, 24
9.23
68,26
61.449
59.59
41.19
15.77
5.84
1.25
0.21
33.77
10.44
rela. improv.
1.29%
4.01%
5.56%
7.34%
14.44%
83.82%
13.14%
Cut
56.54
67
80
1.43
0.04
3.54
Ca-lel)
58.59
51.58
43.
32. 93
22. 27
12.52
5 1..11
2.18
0. 73
0.12
23. 00
4.29
0.3
rela, improv.
3,63%
3.80%
6.29 %
6.92%
11.41%
12.49%
22.87 %
51.05%
135%
200%
6.78%
20.99%
CDRU
56.84
49.82
41 41.50
32.06
20. 80
11.22
4.85
1.51
0.46
0.07
21.91
3.62
Cxx impoov
58.45
61.94
43.9
33.78
22.57
12.80
6.34
2.42
0.65
0.16
23.31
4.49
283%
5.78%
5.36%
30.99%
60.26%
41.30%
129%
24.09%
We test 17 0.1, 0.2, 0.3 our experiments, with the average IoU between the noisy bboxes and their clean versions dropping to 0.833, 0.710, and 0.613, respectively. Examples of the synthesized noisy bboxes can be found in Appendix B.44 As shown Table 3 a-IoU improves the baseline losses (i.e., Lio and CDIOU) considerably in these noisy scenarios. We gain increasing relative improvements from APso APss, which accumulate to more significant improvement mAP7s:95. Note that a-IoU losses also outperform the baselines at AP30 across all noisy scenarios, which is not always the case when bboxes are clean (Table []. Furthermore, a-loU losses are noticeably more robust against more severe noises. For instance, the relative improvement of La.DleU over LI increases from 2.97% 26% to 6.39% according to mAP/mAP AP7//P9999 P75:95 when the noise rate v) rises from 0.1 to 0.3. These results confirm the advantage of a-IoU losses in noisy bbox scenarios.

---

mAP35
MAP
25
15
0
10
a-loU w/o
a-IoU 0.1 noise
a-DioU w/o noise
a--iou
a-loU w/ 0.2 noise
a-lou w/ 0.3 noise
a-DioU w/ 0.2 noise
D-- w/ 0.3 noise
Figure 6: The performance of YOLOv5 models trained using a-loU with different values and evaluated on the clean test set of PASCAL VOC 2007. Black dashed lines denote baselines (i.e., the family of a-IoU with 1) while red dashed lines denote the family of a-loU with 3.
4.4 Sensitivity to power parameter
a
Here, we evaluate the performance of a-loU with varying values (a E [0.5, 5)) via set of experiments with Lo- and -D-D-- The results are shown in Figure YOLOvšs on PASCAL VÓC in both clean and various noisy bbox scenarios. It evident that a-IoU losses with a E [2, perform competitively well across all scenarios, with a = 3 performing the best in most cases. When 3, a-loU losses tend perform worse low APs than the baselines (i.e., a-IoU with a 1), although the performance at high APs gains more improvement. We also test an extreme case with 10. in which the performance drops by 5.61% 21tp//1..99//.....22. 239%%/218// on average compared with a = 3 under noise rates 0/0.1/0.22 1/..2/0..3 respectively. More specifically, it becomes worse than the baselines according to either mAP or mAP7s:95- This indicates that a proper choice of is crucial for a-IoU losses. Our recommendation is to tune E [2, 3] for most applications directly use a when tuning is too expensive. Note that E [3, 4] may be a better choice when high levels of bbox regression accuracy is desired, e.g., mAP is the preferred performance metric. It a better choice for certain applications, although a-loU losses with a < perform consistently worse than the baselines in our experiments.
possible that a <
5 Conclusions
In this paper, we proposed unified formula a-IoU to generalize existing loU-based losses to new family of power loU losses. By modulating the power parameter a, a-loU offers the flexibility to achieve different levels of bbox regression accuracy when training object detector. We analyzed the order preservingness and the loss/gradient reweighting properties of a-IoU, and showed that a-loU improve bbox regression accuracy through up- spiieight the loss and gradient of high IoU objects. Experiments with multiple detection models and benchmark datasets demonstrated that a-loU losses consistently outperform existing loU-based losses, especially at the high Average Precisions (APs). a-IoU has the potential to be widely applied in real-world object detection applications as improves existing loU-based losses, 2) it benefits light models, is extremely advantageous on small datasets, and 4) it is more robust to noisy bboxes. For future work, we will explore new generalization formulas for other netric-derived loss functions such Dice, Hausdorff distance. and Chamfer distance losses.
Societal Impacts
The proposed loss functions can help train siighperffrmance object detectors for impactful applica- tions such as self-driving, face recognition and video surveillance. While not our initial intention, these models could potentially be manipulated by adversaries unauthorized users for malicious purposes. This could compromise the safety or privacy of certain individuals. We believe striet regulations should be established to prevent such illegitimate exploitations.
10

---

References
[1] A. Bochkovskiy, C.-Y. Wang, and H.-Y. M. Liao. Yolov4: Optimal speed and accuracy of object detection. arXiv preprint 200v 2020.
[2] G. E. Box and D. R. Cox. An analysis of transformations. Journal of the Royal Staristical Society: Series (Methodological), 26(2):111244, 1964.
[3] Z. Cai and N. Vasconcelos. Cascade r-cnn: Delving into high quality object detection. In CVPR, pages 6154-6162, 2018.
[4] N. Carion, F. Massa, G. Synnaeve, N. Usunier, A. Kirillov, and S. Zagoruyko. End-to-end object detection with transformers. In ECCV, pages 213-229. Springer, 2020.
[5] K. Chen, J. Pang, J. Wang, Y. Xiong, X. Li, S. Sun, W. Feng, Z. Liu, J. Shi, W. Quyang, et al. Hybrid task cascade for instance segmentation. In CVPR, pages 4974 2019.
[6] K. Chen, Wang, J. Pang. Y. Cao, Y. Xiong, X. Li, S. Sun, W. Feng, Z. Liu, J. Xu, al. Mmdetection: Open mmlab detection toolbox and benchmark. arXiv preprint 17Xiv:9906.07155, 2019.
[7] Z. Dong, G. Li, Y. Liao, F. Wang, P. Ren, and C. Qian. Centripetalnet: Pursuing high- high-quality keypoint pairs for object detection. In CVPR, pages 00519-10528, 2020.
[8] K. Duan, S. Bai, L. Xie, H. Qi, Q. Huang. and Q Tian. Centernet: Keypoint triplets for object 2019.
detection. ICCV, pages
[9] M. Everingham, L.. Van Gool, C. K. Williams, Winn, and Zisserman. The pascal visual object classes (voc) challenge. IJCV, 88(2):303-338, 2010.
[10] J. Gao, J. Wang, S. Dai, L.-J. Li, and R. Nevatia. Note-renn: Noise tolerant ensemble renn for semi-supervised object detection. In ICCV, pages 9508-9517, 2019.
[11] R. Girshick. Fast r-cnn. In ICCV, pages 1440-1448, 2015.
[12] B. Han, Q. Yao, X. Yu, G. Niu, M. Xu, W. Hu, W. Tsang, and M. Sugiyama. Co-tcaching: Robust training of deep neural networks with extremely noisy labels. In NeurlPS, 2018.
[13] J. He, S. Erfani, S. Wijewickrema, S. O'Leary, and K. Ramamohanarao. Learning non-umique segmentation with reward penalty dice loss. IJCNN, pages 1-8. IEEE, 2020.
[14] K. He, G. Gkioxari, P. Dollár, and R. Girshick. Mask r-cnn. In ICCV, pages 2961 2961-2969, 2017.
[15] M. Khodahandeh, A. Vahdat, M. Ranjbar, and W. G. Macready. A robust learning approach to domain adaptive object detection. In ICCV, pages 480-490, 2019.
[16] H. Law and Deng. Cornernet: Detecting objects as paired keypoints. In ECCV, pages 734-750, 2018.
[17] H. Li, T. Fu, J. Dai, H. Li, G. Huang, and X. Zhu. Autoloss-zero: Searching loss functions from scratch for generic tasks, arXiv preprint arXiv:2103. 14026, 2021.
[18] H. Li, C. Tao, X. Zhu, X. Wang, G. Huang, and J. Dai. Auto seg-loss: Searching metric surrogates for semantic segmentation. arXiv preprint arXiv:2010.079.00 2020.
[19] H. Li, Z. Wu, C. Zhu, C. Xiong, R. Socher, and L. S. Davis. Learning from noisy anchors for one-stage object detection. In CVPR, pages (0588-10597) 2020.
[20] Li, C. Xiong, Socher, and S. Hoi. Towards noise-resistant object detection with noisy annotations. arXiv preprint arXiv:2003.01288 2020.
[21] T.-Y. Lin, P. Goyal, R. Girshick, K. He, and P. Dollár. Focal loss for dense object detection. In ICCV, pages 2980-2988, 2017.
[22] T.-Y. Lin, M. Maire, Belongie, J. Hays, P. Perona, D. Ramanan, Dollár, and C. L. Zitnick. Microsoft coco: Common objects in context. In ECCV, pages 740-755. Springer, 2014.

---

[23] P. Liu, G. Zhang. B. Wang. H. Xu, X. Liang, Y. Jiang, and Z. Li. Loss function discovery for object detection via convergence convergence-simulateee simulation driven search. arXiv preprint arXiv:2102.04700, 2021.
[24] W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, and A. C. Berg. Ssd: Single shot multibox detector. In ECCV, pages 21-37. Springer, 2016.
[25] X. Ma, H. Huang, Y. Wang, S. Romano, S. Erfani, and J. Bailey. Normalized loss functions for deep learning with noisy labels. In ICML, pages 6543-6553, PMLR, 2020.
[26] X. Ma, Y. Wang. M. E. Houle, S. Zhou, S. Erfani, S. Xia, S. Wijewickrema, and J. Bailey. Dimensionality-drinn driven learning with noisy labels. In ICML, pages 3355-3364, PMLR, 2018.
[27] N. Natarajan, S. Dhillon, P. K. Ravikumar, and A. Tewari. Learning with noisy labels. NeurlPS, 26: 196-1204, 2013.
[28] M. A. Rahman and Y. Wang. Optimizing intersection-over-uniooo in deep neural networks for image segmentation. In ISVC, pages 234-244. Springer, 2016.
[29] J. Redmon and A. Farhadi. Yolo9000: better, faster, stronger. In CVPR, pages 7263-7271, 2017.
[30] J. Redmon and A. Farhadi. Yolov3: An incremental improvement. arXiv preprint 10X4::8804002765 2018.
[31] S. Ren, K. He, R. Girshick, and Sun. Faster r-cnn: Towards time object detection with region proposal networks. arXiv preprint arXiv:1506.01497, 2015.
[32] H. Rezatofighi, N. Tsoi, J. Gwak, A. Sadeghian, Reid, and Savarese. Generalized intersec- tion over union: A metric and a loss for bounding box regression. In CVPR, pages 658 388-666, 2019.
[33] G. Song, Y. Liu, and X. Wang. Revisiting the sibling head in object detector. In CVPR, pages 11563-11572, 2020.
[34] Z. Tian, C. Shen, H. Chen, and T. He. Fcos: Fully convolutional one-stage object detection. In ICCV, pages 9627-9636, 2019.
[35] L. Tychsen-Smith and L. Petersson. Improving object localization with fitness nms and bounded iou loss. In CVPR, pages 6877-6885, 2018.
[36] K. Wang and L. Zhang. Single-shot two-pronged detector with rectified iou loss. In Proceedings of the 28th ACM International Conference on Multimedia, pages 1311-1319, 2020.
[37] Y. Wang, X. Ma, Z. Chen, Y. Luo, J. Yi, and J. Bailey. Symmetric cross entropy for robust learning with noisy labels. In ICCV, pages 322-330, 2019.
[38] Z. Wang, L. Liu, and D. Tao. Deep streaming label learning. In ICML, pages 9963-9972. PMLR, 2020.
[39] Yu, Jiang, Z. Wang. Z. Cao, and Huang. Unitbox: An advanced object detection network. In Proceedings of the 24th ACM international conference on Multimedia, pages 516-520, 2016.
[40] S. Zhang, C. Chi, Y. Yao, Z. Lei, and S. Z. Li. Bridging the gap between anchor-based and anchor-free detection via adaptive training sample selection. In CVPR, pages
2020.
[41] Y.FF Zhang, W. Ren, Z. Zhang, Z. Jia, L. Wang, and T. Tan. Focal and efficient iou loss for accurate bounding box regression. arXiv preprint 1xX1:22001.0815. 2021.
[42] Z. Zhang and M. R. Sabuncu. Generalized cross entropy loss for training deep neural networks with noisy labels. In NeurlPS, 2018.
[43] 7. Zheng, P. Wang, W. Liu, Li, R. Ye, and D. Ren. Distance-iou loss: Faster and better learning for bounding box regression. In AAAI, pages 12993-13000 13000, 2020.
12

---

[44] X. Zhou, D. Wang, and P. Krähenbühhl. Objects as points. arXiv preprint, arXiv: 19040000 2019.
[45] X. Zhou, J. Zhuo, and Krahenbuhl. Bottom-up object detection by grouping extreme and center points. In CVPR, pages 830-859, 2019.
[46] X. Zhu, W. Su, L. Lu, B. Li, X. Wang, and J. Dai. Deformable detr: Deformable transformers for end-to-end object detection, arXiv preprint arX01:20100001199 2020.
13


Reviews:
[{"review_id": "nOaCuAQUn2b", "reviewer": "Reviewer_Zi6N", "summary": "This paper proposes a novel bounding box regression loss, $\\alpha$-IoU loss, which is a family of power IoU loss. Essentially, the proposed loss is a power transformation of IoU loss. This paper demonstrates that applying such weighting improves detection performance in mAP, increases robustness to noisy label, and is especially powerful in small models.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Overall, I have a very mixed feeling about this paper.\n\nThis paper is well-written. The paper thoroughly studies the proposed loss theoretically and empirically. The quantitative analysis clearly shows that the proposed method outperforms its baseline in mAP and is robust to noisy labels. I appreciate the huge effort this paper made to generalize all IoU variant losses into the family of power IoU loss. I see how this loss can be extremely helpful for training model on noisy label, since it can actively ignore bad labels that give low IoU.\n\nAt the same time, I have a huge concern about the general direction this paper is pursuing - focus more on easy tasks, do easy tasks better. I believe we should aim for the other way around.\n\nThis paper heavily reminds me of focal loss, a power transformation of cross entropy loss. While being the same power transformation, the goal of focal loss is to put less emphasis on easy classes and to focus more on hard / rare classes, which perfectly makes sense as a general goal of perception. In contrast, this paper focuses on optimizing a human-defined, arbitrary evaluation metric, reversing the actual task that we need to solve. Specifically, I would argue that AP95 is an unrealistic evaluation metric that even humans would totally fail. The proposed loss optimizes such an unrealistic evaluation metric, arguing that we should perfect easy tasks and focus less on hard tasks because our evaluation metric doesn’t care about hard tasks anyways. This makes me even question if mAP is the best evaluation metric for object detection. I have a concern that the proposed method may harm recall.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": " - How does AP30-AP95 plot would look like where IoU threshold is on the x axis?\n - How does the proposed method impact precision / recall (not AP)?\n - How does this loss perform on difficult cases such as small or occluded objects?\n\nI believe that answers to the questions above may better demonstrate the limitation of this work.", "review_text": "", "extra_scores": {"ethical_concerns": "N/A", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3hrs"}, "extra_text": {}}, {"review_id": "jg75R14KKLI", "reviewer": "Reviewer_pgM4", "summary": "This paper proposes a very simple extension to the previous IoU losses for object detection bounding regression, by introducing a power term and a power parameter $\\alpha$. The paper gives both theoretical analyses and experimental analyses to the proposed approach. Results on PASCAL VOC and COCO show that the proposed approach outperforms other IoU losses consistently.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "- $Originality:$ This paper proposes an $\\alpha$-loss which introduces a power term and a power parameter $\\alpha$. The proposed approach is interesting.\n\n- $Quality:$ The submission is technically sound. The claims proposed in this paper are well supported by theoretical and experimental analyses. \n\n- $Clarity:$ The paper is well written and easy to follow.\n\n- $Significance:$ This paper shows consistent improvements comparing to different IoU loss baselines. Apart from performance improvements on standard benchmarks, the paper also shows the robustness of the proposed approach on small scale dataset and noisy data. Importantly, the simple extension makes it easier to be adopted by others. There are only a few questions about the experiments. 1) The DETR paper shows improvements by combining IoU loss with L1 loss. It would be interesting to show that the proposed $\\alpha$-loss could also benefit from such combination. 2) It would be interesting to show results of applying the proposed approch to stronger detectors, such Deformable-DETR and Faster/Mask/Cascade-RCNN with stronger CNN backbones. 3) The proposed $\\alpha$-loss has an interesting characteristic that it could let the detector focusing on higher IoU boxes by using a larger $\\alpha$. Cascade RCNN tries to gradually focus on higher IoU boxes by increasing IoU thresholds. It would be interesting to show that the proposed approach could help Cascade RCNN like detectors by gradually increasing $\\alpha$ instead of increasing IoU thresholds. (This could be a future work.)\n\nIn summary, this is an interesting paper with thorough theoretical and experimental analyses. Therefore, I would like to give an accept to this paper.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "hRDeDhLhRIx", "reviewer": "Reviewer_LZaD", "summary": "The paper generalizes existing IoU based losses for bounding box regression into a family of alpha-IoU loss parametrized by one parameter. The paper shows improved object detection performance across multiple datasets and architectures by using different alpha values. ", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality: the paper first proposes to generalize existing IoU losses and studies the effect of hyperparameters.  This leads to improved performance by choosing certain alpha values. \n\nClarity: the paper is well written and easy to follow\n\nQuality: the paper conducts extensive experiments including noisy environment and sensitivity study.  \n\n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes, the paper has addressed possible societal impacts.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "0.5"}, "extra_text": {}}, {"review_id": "el647jZmPsA", "reviewer": "Reviewer_SBt3", "summary": "The paper proposes a generalization of the IoU-based losses which are widely used in applications\nsuch as 2D object detection. This generalization enables the creation of a novel family of losses termed\nalpha-IoU losses which: i) are demonstrated to have the ability to include widely adopted existing losses\ne.g. Generalized IoU ii) give the possibility to create novel losses e.g. L_{\\alpha-IoU}}=1-IoU^{\\alpha} iii)\nare easily tunable with a single hyper-parameter.\nThe authors propose quantitative comparisons wrt existing IoU losses and demonstrate the ability of\ntheir proposed novel family to improve 2D detection performances especially at high-IoUs e.g. with the\nAP75:95. The quantitative analyses have been performed i) on different datasets e.g. PASCAL VOC\nand MS COCO; ii) with different models from high to low capacity e.g. YOLOv5 small and XL; iii) with\nnoisy annotations", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n- The paper proposes a generalization wrt widely used losses in the very popular topic\nrepresented by 2D object detection\n- This generalization is demonstrated to improve the performance of 2d detectors with the only\nchange of the loss function\n- The novel losses are proven to be effective in noisy scenarios, which could be beneficial for\nsettings in which labels contains errors e.g. in semi-supervised or self-supervised settings\n- The novel losses are proven to be effective with models of different sizes, and particularly\nimproving the performance of models with a lower capacity\n- I believe this generalization to be novel and interesting\n\nWeaknesses\n- The proposed family of losses puts more weight on high IoU predictions rather than low IoU\nones. This is reflected in the gradients, as high IoU predictions will have “more gradient” than\nlow IoU ones. But is this a good thing? Why should we down-weight low IoU boxes if those are\nexactly where we should improve the most? Focal loss does a great job in this sense, down-\nweighting what is already classified well and therefore avoiding overfitting, while is it fair to say\nthat alpha-IoU losses encourage overfitting?\n- I believe the authors put too much emphasis on the improvements in the AP75:95 metric but do\nnot adequately analyze the performances wrt other metrics e.g. AP50 or mAP.\n- I believe the results shown on mAP to not adequately sustain the contribution but rather to raise\nsome questions e.g. why, according to the authors, does the AP50 and mAP remain basically\nunaltered? Is the capability of the alpha-IoU limited to improving the IoU on “easier” boxes but\nto be unable to improve the IoU overall and on “harder” boxes? Since mAP values are fair from\nbeing saturated, does the alpha-IoU loss simply help the model to avoid learning “harder” boxes\nand concentrate on the rest?", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "yes", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "1"}, "extra_text": {}}, {"review_id": "Uw77u1D6hyS", "reviewer": "Reviewer_k9Mu", "summary": "This paper extends the existing IoU loss to a new form $\\alpha$-IoU, which helps to improve bounding box regression in learning object detectors. Besides, this loss is also more adaptive and robust to noisy labels. Experiments using various detectors on major detection benchmarks have validated the claims.\n", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Overall, I think this is a great work whose strengths can be summarized as:\n1. The proposed loss formulation is relatively simple and easy to implement. One can easily swap any forms of IoU loss (eg., vanilla IoU, DIoU, GioU, etc.) with the proposed version. \n2. The experimental section is thorough. Both one-stage and two-stage models are tested, and different IoU losses are compared and ablated. The robustness section is interesting.\n3. Sec 3.3 and the corresponding contents in Appendix are very informative and useful. \n4. Writing is mostly clear and related works are properly studied and compared.\n\nWeakness\n1. My biggest concern about this loss ($\\alpha>1$) is: when the training just gets started and loss value is high, the gradient value is small.  However, when the it converges and loss is small, the gradient becomes big. This would cause two bad things: (1) slow down training, and (2) introduce instability to the convergence state. \n2. In practice, It's hard to find the optimal $\\alpha$ for a new dataset or model. Also, it's not clear to me when should we use $\\alpha<1$ or $\\alpha>1$. It would be interesting to see more in-depth results/discussions on this. Theoretical analysis would be better. In fact, I don't even know whether $\\alpha=3$ is dataset specific or something else. \n3. Even though the robustness experiment is interesting, it lacks reasonable explanation or analysis. I'm not very convinced about why the proposed form is more robust. What makes it particularly robust against this kind of noise (inaccurate b-boxes).  \n\nQuestions:\nHow important is it to introduce $\\alpha$ into both the IoU loss and the penalty terms. Which one is more important for these two-term losses?\n\nMisc\n1. I find Fig.3 hard to read. The performance difference is too small on this figure, making the lines stay too close. I would recommend making it simpler and easy to read. \n2. Line44  \"nor decreasing the inference speed is confusing. During inference, most of time we don't compute any loss. Maybe replace with \"training speed\"?\n\nI'm looking forward to the discussion. Please correct my if I understand anything wrong. I'm willing to increase my rating if my doubts are resolved. ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "1. Limitations: Sec.4.4 has discussed the $\\alpha$ issue and provided some insights about how to set it. I also agree that this is arguably the biggest limitation of this work.\n2. Societal Impacts are discussed in the main paper last section, which is reasonable and thoughtful. ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {"ethical_concerns": "I have read the Ethics Guidelines, and I don't find any concerns from this paper.\n1. Potential negative societal impacts: It doesn't involve any social experiments, and doesn't use human-derived data.\n2. General ethical conduct: the dataset used in this work are public ones."}}, {"review_id": "BuylAN5ySH", "reviewer": "Reviewer_TBNp", "summary": "Bounding box regression is a fundamental task in computer vision. The most commonly used loss functions for box regression are the Intersection over Union (IoU) loss and its variants. This paper generalize existing IoU-based losses to a new family of power IoU losses that have a power IoU term and an additional power regularization term with a single power parameter $\\alpha$. Experiments on multiple object detection benchmarks demonstrate the effectiveness  of the proposed loss.", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "1. Detectors should contain recent SOTA methods, like FCOS and ATSS. \n\n2. Improvements is limited on YOLOv5 and DETR.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "1. Experiments should be designed with recent SOTA detectors, like FCOS, Faster R-CNN, Cascade R-CNN and ATSS.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": ["I don’t know"], "time_spent_reviewing": "2 hours"}, "extra_text": {}}, {"review_id": "81d3YJ3-cBi", "reviewer": "Reviewer_BMDv", "summary": "This paper proposes a unified loss formulation for a series of IoU losses. This paper explores its theoretical properties, and conduct several experiments to demonstrate the proposed \\alpha-IoU's effectiveness.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Overall I think this is a good paper. I only have the following minor things for improvement.\n\n1. Plotting the curves of \\alpha-IoU v.s. \\alpha for some IoU values. This could help the choice of \\alpha\n2. In Fig. 3, the legend for different approaches are redundant. For example, the \\alpha-IoU w/o noise appears twice. You can remove one of them.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3 hours"}, "extra_text": {}}]

Author rebuttal:

Meta-review: Reviewers agreed that this is a solid paper that deserves acceptance. Authors are highly encouraged to address the key comments reported by reviewers as well as to implement all the improvements (as indicated by authors in the rebuttal) in the final camera-ready version. 
Decision: Accept (Poster)