Learning High-Precision Bounding Box for Rotated Object Detection via Kullback-Leibler Divergence
Authors: Xue Yang
Xiaojiang Yang
Jirui Yang
Qi Ming
Wentao Wang
Qi Tian
Junchi Yan
Venue: NeurIPS 2021 Poster (2021)
Keywords: Rotated Object Detection
High-Precision
Kullback-Leibler Divergence

Abstract: Existing rotated object detectors are mostly inherited from the horizontal detection paradigm, as the latter has evolved into a well-developed area. However, these detectors are difficult to perform prominently in high-precision detection due to the limitation of current regression loss design, especially for objects with large aspect ratios. Taking the perspective that horizontal detection is a special case for rotated object detection, in this paper, we are motivated to change the design of rotation regression loss from induction paradigm to deduction methodology, in terms of the relation between rotation and horizontal detection. We show that one essential challenge is how to modulate the coupled parameters in the rotation regression loss, as such the estimated parameters can influence to each other during the dynamic joint optimization, in an adaptive and synergetic way. Specifically, we first convert the rotated bounding box into a 2-D Gaussian distribution, and then calculate the Kullback-Leibler Divergence (KLD) between the Gaussian distributions as the regression loss. By analyzing the gradient of each parameter, we show that KLD (and its derivatives) can dynamically adjust the parameter gradients according to the characteristics of the object. For instance, it will adjust the importance (gradient weight) of the angle parameter according to the aspect ratio. This mechanism can be vital for high-precision detection as a slight angle error would cause a serious accuracy drop for large aspect ratios objects. More importantly, we have proved that KLD is scale invariant. We further show that the KLD loss can be degenerated into the popular Ln-norm loss for horizontal detection. Experimental results on seven datasets using different detectors show its consistent superiority, and codes are available at https://github.com/yangxue0827/RotationDetection.

Learning High-Precision Bounding Box for Rotated Object Detection via Kullback-Leibler Divergence
Xue Yang', Xiaojiang Yang', Jirui Yang", Qi Ming', Wentao Wang', Qi Tian", Junchi Yan
Department of Computer Science and Engineering,
MoE Key Lab Artificial Intelligence, Al Institute, Shanghai Jiao Tong University
University of Chinese Academy of Sciences
s Beijing Institute of Technology *Huawei Inc.
-yangxue-2019-jjae yangxiaojian jang, wwt117, vanjunchi] jt. cn (yangjirui123, chaser (mm.mmmnnaaall com tian. qi gi1@huawei.co com
Abstract
Existing rotated object detectors mostly inherited from the horizontal detection paradigm, as the latter has evolved into a well-developed area. However, these detectors are difficult to perform prominently in high-precision detection due to the limitation current regression loss design, especially for objects with large aspect ratios. Taking the perspective that horizontal detection is special case for rotated object detection, in this paper, we are motivated to change the design of rotation regression loss from induction paradigm deduction methodology, in terms of the relation between rotation and horizontal detection. We show that one essential challenge is how to modulate the coupled parameters in the rotation regression loss, as such the estimated parameters can influence to each other during the dynamic joint optimization, an adaptive and synergetic way. Specifically, we first convert the rotated bounding box into a 2-D Gaussian distribution, and then calculate the Kullback-Leible Divergence (KLD) between the Gaussian distributions as the regression loss. By analyzing the gradient of each parameter, we show that KLD (and its derivatives) can dynamically adjust the parameter gradients according the characteristics of the object. For instance, it will adjust the importance (gradient weight) of the angle parameter according to the aspect ratio. This mechanism can be vital for high-precision detection as a slight angle error would cause a serious accuracy drop for large aspect ratios objects. More importantly, we have proved that KLD is scale invariant. We further show that the KLD loss can be degenerated into the popular 4 -oorm loss for horizontal detection. Experimental results on seven datasets using different detectors show its consistent superiority, and codes are available at
github...... http:///...auuuee////////aaaaata
Introduction
As fundamental building block for visual analysis across acrial images, scene text etc., rotated object detection has recently been developed rapidly 0
6 which benefit themselves Specifically, many works build themselves upon the previously established horizontal detection pipeline from an inductive perspective, as shown in Figure [(a] However, these detectors are often unable to cope with challenging scenes well due to the limitations of current regression loss, such large aspect ratio objects, dense scenes, etc., resulting in obvious disadvantages in high- detection.
2
from the well-established horizontal detection approaches 7 S 9
12 1 14
15]
Part of the work was done during an internship at Huawei Inc.
t Cooresponndecce author is Junchi Yan.
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

horizontal
Induction
rotation
horizontal
Deduction
rotation
special to general
general to special
optimize
reg. loss
optimize
optimize
reg. loss
optimize
L,,nom (x,y. w.
norm
norm
KLD
(x,y,,, w.,, h. 8)
(x. y.
(1. ΣΣ
(a) Previous methods follow the induction paradigm (b) Our proposed method adopts deduction methodol- from special horizontal to general rotated detection.
ogy from general rotated to special horizontal detection.
Figure 1: Methodological road-map difference between horizontal detection (special case) and 15] and the proposed method.
rotation detection (general case) in the previous methods 12
In this paper, we take step back, and aim to develop (from a deductive perspective) unified regression framework for rotation detection and its special case: horizontal detection. In fact, our new framework enjoys coherent property that it can be degenerated into the current commonly used regression loss (e.g. -nrmm) special cases (horizontal detection), as shown in Figure l(b)
For a devising a rotation regression loss for high-precision rotation detection, one important observa- tion is that the importance of different parameters different types of objects can vary. For example, the angle parameter (0) and the center point parameter (æ, y) are important for large aspect ratio objects and small objects, respectively. another word, it is conjectured that regression loss should be self-modulated during the learning process and calls for more dynamic optimization strategy.
Inspired by the above ideas, first convert the rotated bounding box B(x, y, h, w, 0) into 2-D Gaussian distribution ((,, ). As a standard distance metric, we then use the Kullback-Leible Divergence (KLD) [6] calculate the distribution distance between the predicted bounding box and ground truth as the regression loss. We compare KLD with Smooth L1 loss (7] and another distance metric, Gaussian Wasserstein Distance (GWD) (5) 17]. and find that KLD has a more complete parameter optimization mechanism. In particular, by analyzing the gradient of the parameters during learning, we show that the optimization of one parameter will be affected by other parameters (as the gradient weight). It means that the model will adaptively adjust the optimization strategy given a specific configuration of an object for detection, as shown can lead excellent performance in high-precision detection. In addition, KLD is proven scale invariant, which is an important property that Smooth LI loss and GWD do not possess. As the horizontal bounding box a special case rotated bounding box, we show that KLD can also be degenerated into the 4, norm loss as commonly used existing horizontal detection pipeline. The highlights of this paper are four-folds:
1) Differing from the dominant existing practices that build rotation detectors heavily upon the horizontal detectors, we develop new rotation detection loss from scratch and show that it coherent with existing horizontal detection protocol in its degenerated case for horizontal detection.
2) To achieve computing the difference for each physicaaly-mmaninghhi parameter related to the bounding box which different scales and units, innovatively convert the regression loss of rotation detection into the KLD of two 2-D Gaussian distributions, leading to a clean and coherent regression loss.
principled measurement between the prediction and ground truth, instead of
3) Through the gradient analysis of each parameter in KLD, we further find that the self-modulated optimization mechanism of KLD greatly promotes the improvement of high sighiprecision detection, which verify the advantage of our loss design. More importantly, have theoretically shown (in appendix) that KLD is scale invariant for detection, which is crucial for the rotation cases.
4) Extensive experimental results on seven public datasets and two popular detectors show the effectiveness of our approach, which achieves new state-of-the- performance for rotation detection. The source codes [18] are made public available.
2 Background
We first generally discuss the related works on both horizontal and rotated object detection. Then we summarize the current design paradigm of rotation regression loss from two kinds of methodologies,

---

BALELAR
Glaagroo Internets
Abr wort
Glasgov Internsitc ma
Alrport
Glasgnow Interasaic Alrpart
Figure 2: Visual comparison between Smooth L1 loss (left), GWD (middle) and KLD (right).
as shown in Figure [ one is inductive that tries to develop the general rotation detection from the special and classic horizontal detection pipeline. While the other is deductive that aims to devise general rotation detection pipeline with horizontal detection as its special case.
2.1 Related Works
Horizontal object detection. Horizontal object detection which covers most existing detection literature, normally uses a horizontal bounding box to represent the object. The mainstream classical object detection algorithms can be roughly divided according to the following standards: Two- or Anchor-based
78
[1] or Single-stage [00 19 20
object detection, Anchor-free [21] 22 23
8,
9
10
object detection and CNN 8
10 21) or Transformer-basee
[24 25 object detection.
Although the pipelines may vary. the mainstream regression loss often uses the popular L -nom
loss (such smooth L1 loss) or loU-based loss (such GloU (26), and DIoU (27)) These above- mentioned detectors have also been widely used in other scenarios and have achieved satisfactory performance. However, horizontal detectors do provide accurate orientation and scale information.
28 are mainly driven by adapting the horizontal object detectors with rotated bounding boxes to represent multi-oriented objects. To accurately predict the rotated bounding box, most rotation detection methods extend the 1,,norm 22
Rotated object detection. Recent advances in rotation detection 3
31 used in horizontal detection, construct differentiable approximate 32 From scratch, we try to change the design rotation regression loss from induction paradigm to deduction methodology, which in fact is generalization the horizontal case.
15
29
30
IoU loss 3
5
In the following, we describe the existing works from the induction and deduction methodologies.
2.2 Inductive Thinking Loss Design: from Special Horizon General Rotation Detection
Regression loss is a vital part of most current object detection algorithms. For horizontal bounding [I] mainly outputs four items for location and size:
box regression, the model 7 8 10
Ep
Up
Ha
In
1h
In
hip
(1)
10
has
Win
to match the four targets from the ground truth
Na
Vc
Ya
In
We W.
1
In
hec
Pen
Pr..
where æ, y, h, w denote the center coordinates, height and width, respectively. Variables Wt Day Tp are for the ground-truth box, anchor box, and predicted box, respectively (likewise for y, w, h).
3

---

Extending the above horizontal case, existing rotation detection models [1
12
13
14
15
also use
regression loss which simply involves an extra angle parameter 0:
f f(ep 0a), fa = f(; - 0a)
(3)
where /()) is used to deal with angular periodicity, such as trigonometric functions, modulo, etc. The overall regression loss for rotation detection is:
Livy = tn-norm (Atz: Aly. Atw, Al, Ala)
(4)
where At the - -t Az Ay E2y - Ay At = tt ln(wup/ (/)p/W)), At = - t/ In(hp// /1),, and Ato
t A6.
can be seen that parameters are optimized independently, making the loss (or detection accuracy) sensitive to the under-fitting of any of the parameters. This mechanism is fatal to high-precision detection. Taking the left side of Figure as an example, the detection result based on the Smooth loss often shows the deviation of the center point angle. Moreover, different types of objects have different sensitivity to these five parameters. For example, the angle parameter is very important for detecting objects with large aspect ratios. This requires to select appropriate set of weights given specific single object sample during the training, which nontrivial or even unrealistic.
2.3 Deductive Thinking of Loss Design: from General Rotation to Special Horizon Detection
To break the original inductive design paradigm, we adopt deductive paradigm to construct more accurate rotation re- gression loss. Here we rephrase the main idea in the recent bounding box B(x, y, h, w. 0) into 2-D Gaussian ((,, Σ), as il- lustracted in Figure 3 Then the distance between two Gaussian is calculated as the final loss. Specifically, the conversion is:
777
work [5], which converts a aebitrary-oriente
Figure 3: Top: rotated box Bax, V. h, 10 0). Bottom: 2-D Gaussian dist. N N(,,, E).
(x==x,))
Σ 1/2 -RAR sin 0
cos
sin 0 cos
)
cos sin
cos
cos? + h sin? 2 0
w- cos 0 sin 0 sin 0 + cos? 0
cos 0 sin 0
(5)
where R represents the rotation matrix, and A represents the diagonal matrix of eigenvalues.
The recent work (5] analyzes that the introduction of (M, Σ) can solve the inconsistency between metric and loss, boundary discontinuity and square-like problem. On this basis, we further study how to design high-precision detection regression loss through new parameter space. Our view is that the self-modulated mechanism is positively correlated with the final high-precision performance.
Gaussian Wasserstein Distance. The Wasserstein distance 5 17
between two probability measures
X, Ngl1pp Σp) and X, Ni(p,, Σ,) expressed as:
(N |k4p 1el13 Tr(E, + E. 2 / EE
(6)
center distance
coupling temus about Agp. Wpp and By.
Eq. 6 shows that the Gaussian Wasserstein Distance (GWD) is mainly divided into two parts: the distance between the center points (I. y) and the coupling terms about h, w and f. Accordingly, the regression loss based on GWD can be regarded as a semi-coupled loss. Although GWD can greatly improve the performance of high-precision rotation detection due the coupling between part of the parameters, the independent optimization of the center point make the detection result slightly shifted (see Figure Note that GWD is not scale invariant, which is not detection friendly.
When all the boxes are horizontal (0 = 09), ). Eq. 6 can be further simplified:
(N,,,
[4, 1eel2 + |ز// 1
=(x, 2z.)² (2p y1)² (ww, we) (hi hv)) /4 =lg-norm AA., Ay, Aw/2 A 22/

---

where || |F is the Frobenius norm. Although Eq. 7 can still be used as the regression loss of horizontal detection, Eq. and 7 are not completely consistent.
Although GWD scheme has played a preliminary exploration of the deductive paradigm, it does not focus achieving high-precision detection and scale invariance. In the following, will propose our new approach based on the Kullback-Leible divergence (KLD) [16].
3 Proposed Approach
Kullback-Leine Leibler Divergence. To explore the more appropriate regression loss, we adopt the Kullback-Leible divergence (KLD) [6]. Similarly, the KLDD between two 2-D Gaussian is:
Dn((I))) (pp pc) E. (Me m.)
TrE, Epp) In
(8)
team about upp and
coupling terms about fip. spp and Bp
or
Ep Σn
DM(Ni)) = (pp - Mc) Ex "pp(pp mi) + Tr E.)
In
-1
chain coopling of all puramesers
can be seen that each item in D (NG (W,)) ) is composed of partial parameter coupling, which makes all parameters form a chain coupling relationship. In the optimization process of the KLD-base detector, the parameters influence each other and are jointly optimized which make optimization mechanism of the model is self-modulated. In contrast, DR (NGlINN) and GWD are both semi- coupled, but DRa (N)) has a better central point optimization mechanism.
Although KLD is asymmetric, we find that the optimization principles of these two forms are similar by analyzing the gradients of various parameters and experimental results. Take the relatively simple Dar (Na (l)))
as an example, according to Eq. 5 each item of Eq. 8 can be expressed as
(1p- p4)² Σ, ( (c))
(Ax cos Ay sysn 0.)
4 (Ay cos 0
Ar sin 0.)² 0.)
(10)
w
h2
Tr(E, Ep) Tr(ET'Ep)
h3 AA
sin
Wp 12
sin 2
0A
+
cos
20 +
cos
AA
(11)
w
w
Σ,
h2
w
In
= In
+
In
(12)
Epl
h
where Ar = xp 22, Ay = Up Vt. A = 02 tr.
Analysis of high-precision detection. Without loss of generality, set 0, 0%, then
DDw( 0up
( 4 Ar,
Av
(13)
wm
The weights 1 1/wž and 1/h? will make the model dynamically adjust the optimization of the object position according to the scale. For example, when the object scale is small
edge is too short, the model will pay more attention to the optimization of the offset of the corresponding direction. For this kind of object, slight deviation on the corresponding direction will often cause sharp drop in loU. When 0g * 0", the gradient of the object offset (Ar and Ay) will be dynamically adjusted according to the 9 for better optimization. In contrast, the gradient of the center point in GWD and Ax, no. Ay)T The former cannot adjust the dynamic gradient according to the length and width of the object. The latter is based on the length and width of the anchor (wa, ha) adjust the gradient instead of the target object (we, which
L2-norm are @D..(Hn)
(2Ar, 2Ay) T and @Lz(Hz)
djt
is almost ineffective for those detectors 3 13 15] 28 29 33 34) that use horizontal anchors for rotation detection. More importantly, they are not related to the angle of the target object. Therefore, the detection result of the GWD- based and L,, -oom models will show a slight deviation, while the detection result of the KLD-based model is quite accurate, as shown in Figure
For htp and Wp, we have
@Dui(Ep)
h2 h
2 A0
h
sin 2 A0 1,
0D (D,)
w coS A0 + sin A0 - (14)
Mi
2
cos
0 In ltp
i In Wp
5

---

On the one hand, the optimization of the hp and up is affected by the AG. When A = 0% (Dz((Ep) h² kl R (Dx wy wf L, which means that the smaller targeted height width leads heavier penalty on its matching loss. This desirable, as smaller height or width needs higher matching precision. On the other hand, the optimization of AO is also affected by hp and Up:
(Dn)
W
sin 2A0 h
(15)
88p
wi
when "y We: hip h2s then (D,(EE)) wi N sin 2A0 Z sin 220, the condition for
"
2
80,
the equality sign is h4 wy- This shows that the larger the aspect ratio the object, the model will pay more attention the optimization of the angle. This is the main reason why the KLD-based model has a huge advantage in high-precision detection indicators as slight angle error would cause serious accuracy drop for large aspect ratios objects. Through the above analysis, we find that when one of the parameters is optimized, the other parameters will be used as its weight to dynamically adjust the optimization rate. In other words, the optimization of parameters no longer independent, that is, optimizing one parameter will also promote the optimization of other parameters. The optimization of this virtuous circle is the key KLD as an excellent rotation regression loss. In addition, D& (NG has similar properties, refer to appendix for details.
Scale invariance. For full-rank matrix M, M # 0, we have (N = DRI(N, (1Vg),
where X, = MX, ~ (Mpp. MΣ,M") X, = MX, ~ (M ME, Therefore, the affine invariance (including scale invariance when M kI. where denotes identity matrix) of KLD can be proven (see proof in appendix). Compared with L,, -norm and GWD, KLD is more suitable for replacing the lon-differentiable rotated IoU loss for its consistency with detection metric.
Horizontal special case. For horizontal detection, combine Eq.
8 to Eq. 12
we have
1
4A2r
4A2 h
In
20
w
(16)
= 2222-norm (At,, At,) 1 norm Mt Atr.) 12-morm(
)-1
Atw
Ath.
where the first two terms of Eq. 16 are very similar to Eq. 4 and the divisor part of the two terms x and y is the main difference
vs. Wa ).
Az).
Variants of KLD. We have also introduced some variants
35 36 of KLD to further verify the influence asymmetry on rotation detection can be ignored. The variants mainly including
D maLminmmaz) (N (NG]LNN) min(max) (AGIAA))))) (NGIIN))))
D (N,(N) - DE Aill Nr N
+ Dai [N No +.2
(17)
2 Daer (Va(A)) + (N,
2
Rotation regression loss. The whole training process of detector is as follows: i) predict offset (t2, #P, tw. th, th)); ii) decode prediction box; iii) convert prediction box and target ground- truth into Gaussian distribution; iv) calculate KLD of two Gaussian distributions. Therefore, the inference time remains unchanged. We normalize the distance function as our final regression loss Lreg:
Greg =
T T(D T21
(18)
where f(.) denotes non non-linear function to transform the distance D to make the loss more smooth and expressive. this paper, we mainly use two nonlinear functions, (Dr((D) and In(D + 1). The hyperparameter modulates the entire loss. The multi-task loss is:
N A1
Ny.A Σ Lreg (bn gt...
Aa N
Σ (pn, tw.)
(19)
where Npns and N indicate the number of positive and all anchors. b, denotes the n-th bounding box, gh,, is the n-th target ground-truth. i,, denotes the label of n-th object, pa is the n-th probability distribution of various classes calculated by sigmoid function. The hyper-parameter Å1, X2 control the trade-off and are set to {2, 1} by default. The classification loss Lote is set as focal loss [10].

---

Table 1: Ablation study of the loss form and hyperparameter on HRSC2016.
(a(f(Du).7)
Loss
Dur
[(Dw)
=
==3
((D sgrt(Det) f(Da) DD +
82.96
84.85
84.15
75.23
73.32
0.20
83.23
85.25
83.63
80.79
73.44
Table Ablation of different KLD-based regression loss form.
The based detector is RetinaNet.
Dataset
(,,,)
Du men (V.l))
Dat maz (nGa(N)
Defffrru (N,])
DOTA-v1.0
70.17
70.64
70.71
70.55
69.67
70.56
HRSC2016
82.83
83.82
83,60
82.70
84.06
83,66
Table Ablation study of normalization. The based detector is RetinaNet.
Loss
Norm by Eq. 18
HRSC2016
DOTA-v1.0
Hincanso
Hinaans
Hmcaatapss
AP'o
Smooth Li
w/ 78.99
43.12
43.47
64.95
w/o
84.80
48.42
47.76
65.73
4 Experiment
4.1 Datasets and Implementation Details
Our experiments are conducted over variety of datasets, including three large-scale public datasets
for aerial images i.e. DOTA 37 UCAS-AOD 38 HRSC2016 [39], as well as scene text dataset
ICDAR2015
MI.T and MSRA-TD500
DOTA is one of the largest dataset for oriented object detection in aerial images with three released versions: DOTA-v1.0 DOTA-v1.5 and DOTA-v2.0. DOTA-v1.0 1.0 contains 15 common categories, 2,806 images and 188. 2882882 instances. The proportions of the training set, validation set, and testing set in DOTA-v1.0 are 1/2, 1/6, and 1/3, respectively. contrast, DOTA-v1.5 uses the same images as DOTA-v1.0, but extremely small instances (less than 10 pixels) are also annotated. Moreover, new category, containing 402,089 instances total is added in this version. While DOTA-v2.0 contains 18 common categories, 11,268 images and 1,793,658 instances. Compared to DOTA-v1.5, it further includes the new categories. The 11,268 images DOTA-v2.0 are split into training, validation, test-dev, and test-challenge sets. We divide the images into 600 x 600 subimages with an overlap of 150 pixels and scale it to 800 x 800, line with the cropping protocol in literature 28].
UCAS-AOD contains 1,510 aerial images of approximately 659 x 1, 280 pixels, with two categories of 14,596 instances in total. In line with 31 37 we randomly select 1,110 for training and 400 for testing. HRSC2016 contains images from two scenarios including ships on sea and ships close inshore. The training, validation and test set include 436, 181 and 444 images.
ICDAR2015, MLT and MSRA-TD500 are commonly used for oriented scene text detection and spotting. ICDAR2015 includes 1,000 training images and 500 testing images. ICDAR2017 MLT is multi-lingual text dataset, which includes 7, 7,200 training images, 1,800 validation images and 9,000 testing images. MSRA-TD500 dataset consists of 300 training images and 200 testing images.
We use Tensorflow 43 to implement the proposed methods on server with Tesla V100 and 32G memory. The experiments are all initialized by ResNet50 44] by default unless otherwise specified. Weight decay and momentum are set 0.0001 and 0.9, respectively. We employ MomentumOptimizer over 8 GPUs with total of images per minibatch (1 image per GPU).
All the used datasets are trained by 20 epochs in total, and the learning rate is reduced tenfold at 12 epochs and 16 epochs, respectively. The initial learning rate is set to 5e-4. The number of image iterations per epoch for DOTA-v1.0 1.0, DOTA-v1.5, DOTA-v2.0, UCAS-AOW HRSC2016, ICDAR2015, MLT and MSRA-TD500 are 54k, 64k, 80k, 5k, 10k, 10k, 10k and 5k respectively, and doubled data augmentation (include random rotation, flipping, and graying) or multi-scale training used.
4.2 Ablation Study and Further Comparison
Regression loss form and hyperparameter. Table I compares three forms of KLD-based regression loss on HRSC2016, including DRI. (Dxx and Creg (f (DRi), r). Due to extreme sensitivity to large
7

---

Table 4: figh-precision detection experiment under different regression loss. RR' 'F' and 'G' indicate random rotation, flipping, and graying, respectively. The resolution of HRSC2016, MSRA-TD500 and ICDAR2015 are 500 x 500, 800 x 1, 000 and 800 x 1, 000, respectively.
Method
Dataset
Data Aug.
Reg. Loss
Elmean./APp /AP
Hmeany./APPu
Hmeany:/APpy
Hmesnss/APis
Smooth
84.28
74.74
12.56
RetimaNet
GWD
85.56 (+1.28)
84.04 (49.30)
60.31 (+11.9)
(64.58)
52.8 (45.13)
HRSC2016
R-F+G
Smooch
KLD
87.45 (43.17)
88.32
86.72 (611.98)
19.01
(42397)
41422
27.68 (+15.12)
57.80 (+30,04)
46.18
R'Det
(40.11)
88.89 (49,88)
(422.46)
15.02 (410.44)
(49,89)
KLD
89.97 (3.45)
89.73 (+10.2))
77.38 (433.96)
25.12 (420.44)
61.40 (+15.22)
Smooth LI
70.98
62:42
36,73
12.56
37.89
MSRA-TD500
R+F
GWD
(45.78)
68.58 (46.16)
44.21 (47.48)
17.75 (45.19)
43.62 (65.73)
KLD
(47.66)
(310.23)
45.24 (67.35)
Smooth LI
69.78
61.15
36.97
8.71
37.73
RetimaNet
F
GWD
74.29
68, 34 (+4.9)
43.39 (+6.42)
10.50 (+1.79)
41.68 (+3.95)
KLD
75.32 (45.54)
(45.79
44.46 (+7.49)
10.70 (+1.99)
42.68 (-4.95)
Smooth
11:39
R+F
GWD
76.15 (+1.2))
71,26 (+1.80)
45.59 (+3.57)
11,65 (+0.00)
43.58 (-1.60)
ICDAR2015
Smooth I.
KLD
77.92 (43000)
74.28
72.77 (+3.31)
68.12
43.27 (+1.25)
35.73
11.19 (00 50)
8.01
43.65 (-1.67)
39.10
F
(+0.24)
(+4.11
(-1.70)
R'Det
KLD
77.72
71.99 (43.7)
43.95
10.43 (42.22)
43.29 (44)19)
Smooth L.1
15.33
69.69
37.69
9.03
40.36
R+F
GWD
77.09 (41.56)
71.52 (41283)
41.08 (43.29)
10.10 (41.07)
42.17 (+1L6))
(+4.6))
73.30 (4366)
(4582)
(+1.58)
43.61 (+3.05)
errors, the performance of D& is extremely poor, only 0.20%. Through simple nonlinear linear transformation, the performance be increased to 82.96% and 83.23% corresponding to sqrt and log. We further perform a detailed hyperparameter experiment on the loss Creg proposed in this paper, and the performance reaches the optimal when 85.25 %. Keeping the same loss pattern, we compare six KLDDbbasdd distance functions in Table 2 and conclude that the asymmetry of KLD does not have much impact performance. In subsequent experiments, we use Lrey (log (NV(l))))) 1) as the basic setting.
1, (Dx1) log(Dwr + 1). about
Ablation study of normalization. As mentioned above, the use of Eq. 18 is to smooth its excessively rapid growth trend and play a role of normalization. This extra normalization questions if the KLD is actually contributing or simply produces noise in the results. In order to further prove that our method is indeed effective, we also perform a normalization operation on the Smooth LI loss to eliminate the interference caused by normalization. As shown in Table 3 there is significant drop in performance after using the normalization. The above experimental results prove that the effectiveness of KLD does not come from Eq. 18
High-precision detection experiment. We expect that the designed rotation regression loss can show advantages in high-precision detection. Table shows the comparison of the high-precision detection results of three different regression losses using Smooth L1, GWD and KLD on different datasets and different detectors. For the HRSC2016 6 dataset containing a large number of ship with large aspect ratios, GWD-based RetinaNet has a 11.89 % improvement over Smooth L.1 on AP75; KLD even gets 23.97% gain. Even with stronger Det detector, KLD and GWD still increased by 33.96% % and 22.46 % in AP7s. and 15.22% and 9.89 % in AP50-95- The same experimental conclusion are also reflected in the other two scene text datasets MASR-TF500 and ICDAR2015, which is KLD > GWD > Smooth L1. In general, the self-modulation optimization mechanism has a significant help for high-precision detection. For more intuitive comparison, we visually compare these three regression losses, as shown in Figure 2 Since the center point (x. y) parameters in Smooth L1 Loss and GWD independently optimized, their prediction results are slightly shifted. In contrast, the KLD-based prediction results are closer to the object boundary and show strong robustness in dense scenes. Similarly, GWD--bsed or KLD-based model has more accurate angle prediction capabilities than Smooth L.-based model due to their angle parameters (0) are not independently optimized.
Ablation study on more datasets. To make the results more credible, we continue to verify on the other five datasets, as shown in Table The improvement of KLD on the three data sets of MLT, UCAS-AOD and DOTA-v1. 1.0 O is still considerable, with increase of 9.17% % 1.58% %, and 5.55 % respectively. Note that for DOTA-v1.5 and DOTA-v2.0, which contain a large number of small objects (less than 10 pixels), KLD has achieved significant gains of 3.63% % and 3.53%.
Comparison of peer methods. Table compares the six peer techniques, including IoU-Smooth LI Loss 3 Modulated loss 45 RII 34
CSL 47 DCL 46). and GWD (5 on DOTA-v1.0 .O. For fairness, these methods are all implemented on the same baseline method, and are trained and tested under the same environment and hyperparameters. We detail the accuracy of the seven categories,
8

---

Table 5: More ablation experiments on other datasets.
Method
Loss
MLT
UCAS-AOD
DOTA-vL.0
DOTA-vI.5
DOTA-v2.0
Smooth LI
48.42
94,36
65.73
58.87
44.16
RetinaNet
GWD
54.58 (66.16)
95.44 (10.55
68.93 (+3.20)
60.03 (+1.16
46.65 (+2.49)
KLD
57.59 (+9.17)
96.14 (+1.58)
71.28 (+5.55)
62.50 (+3.63)
47.69 (43.53)
Table 6: Accuracy comparison between different rotation detectors on DOTA dataset. and represent the large aspeet ratio object and the square-like object, respectively. The bold red and blue fonts indicate the top two performances respectively. Dae and Die represent OpenCV Definition (0 E 909, 09)) and Long Edge Definition (0 E )00)) of RBox.
Baseline
Method
BRR
SVI
531
trannoitett
11A
RAV
T-APs3
AP3)
AP1s
<1.0 traineee
AP
APrap0
AP/o
+1.5
AP33
x2.0
Pac
42.17
05.03
31:11
72.51
33:24
75.35
02:00
05.775
03.73
06.70
32.31
34.30
33.57
<4.10
Di.
38.31
60.48
39.77
68. 29
51.28
75.60
60.02
58.11
64.17
62.21
26.06
31.49
16.10
43.06
JoU Smooth Li
D
64.32
63.03
51.25
12.78
56.21
71.98
63:22
61:26
06.99
64.61
34.17
36.23
59.16
45.31
Modulaeed Loos
33
42.92
67.92
52.91
72.67
53.64
88.22
58.21
61:21
66.11
63.50
33.32
34.61
57.75
45.17
RecinaNet
Modalased Loos
RIL
54
Quad.
Quad.
43.21
40.81
67.63
70.78
54.70
35.45
72.68
72.42
55.49
61.99
75.09
79.72
64.75
02.08
62.09
63:45
67.20
06.06
65.15
64.07
40.59
40.98
29.05
29.13
61.42
59.91
45.35
46.71
CSL
Dr
42.25
68.28
51.51
72.35
53.10
75.59
38.99
60.80
67.38
64.40
32.58
35.04
19.55
43.34
DCL
D.
41.50
65.82
56.27
75:80
$4.30
79.02
60.25
61.55
67.39
65.93
35.56
36,71
59.38
45.46
GWD
D=
44.07
71.92
62.56
77.94
60:25
79.64
63.52
68.70
68.93
65.44
38.68
$8.71
60:00
48.65
KLDD
44.00
74.45
72.48
84.30
65.54
96.03
65.05
09:41
71.25
68.14
44.18
42:15
62.50
47.09
D
14.15
75.05
72.88
3014
36.29
K2.33
01001
65.21
70.60
07.18
35.11
35.25
03.91
43.33
R'Des
(BCL))
GWD 5
10
D
D.
46.73
46.84
74.87
75.84
74.96
78
35.70
86.71
57.72
62.69
X3,19
84.06
61.12
63.77
69.70
70.60
71.56
71.21
67.45
69.28
35.44
43.35
37.54
41.56
61.98
63.22
49.25
48.71
KL.D
D
45.34
75.09
78.88
86.53
58
82.08
61.51
71.13
71.75
68.87
44.48
42.11
65.15
50,90
including large aspect ratio (e.g. BR, SV, LV, SH, HA) and square-like object (e.g. ST, RD), which can better reflect the real-world challenges and advantages of our method. Without bells and whistles, the combination of RetinaNet and KLD directly surpasses R Detet (71.28% vs. 70.66% % in AP and 69.41 % 68.31 % in 7-)P Even combined with Detet, KLD can still further improve performance of the large aspect ratio object (2.82% % in 7-APs0) 50) and high-precision detection (6.07% % in AP7s and 3.65% 50:95). KLD-based method shows the best performer in almost all indicators. Similar conclusions can still be drawn on the more challenging datasets (DOTA-v1.5 and DOTA-v2.0), which contain more data and tiny object (less than 10 pixels).
Horizontal detection verification. As analyzed by Eq. 16 KI.D can be degenerated into the common regression loss in horizontal detection task. Table compares the regression loss Smooth LI and
loU/GIoU for horizontal detection with the proposed regression loss KLD on MS COCO 48 dataset. The results show that our KLD is not worse than other losses on the Faster RCNN (8), RetinaNet [IO] and FCOS (21) and even has an improvement of 0.6% on RetinaNet. The ground truth for rotation detection is the minimum circumscribed rectangle, which means that ground truth can well reflect the true scale and direction information of the object. The "horizontal special case" described in this paper also meets the above requirements, the horizontal circumscribed rectangle is equal to the minimum circumscribed rectangle at this time. Although the ground truth of the COCO is horizontal box, it is not the minimum circumscribed rectangle, which means that it loses the direction information and accurate scale information of the object. For example, a baseball bat placed obliquely in the image, the height and width of its horizontal cireumscribed rectangle do not represent the height and width of the object itself. This causes that when KLD is applied to the COCO, the optimization mechanism of KLD that dynamically adjusts the angle gradient according to the aspect ratio is meaningless, which affects the improvement of the final performance. In general, this is defect in the dataset annotation itself, not that KLD is not good enough. fact, it is inappropriate to use the COCO to discuss 0 = 0%, because the COCO discards 0 parameter. In addition, 0 = 0 describes the instances in the horizontal position, but not mean all instances of the dataset are in a horizontal position. This paper uses COCO to discuss the "horizontal special case" to express that even if the dataset has certain labeling defects, KLD can have certain effects. After all, it is difficult to observe the performance improvement of all horizontal objects on the rotating dataset.
4.3 Comparisons with the State-of-the-Art Methods
The evaluation is performed on the DOTA, which contains a considerable number of categories, complexity scenes. Our single-scale model RetinaNettKLLD-RS. and D achieve 75.28% % and 77.36 % respectively. They outperform multi-scale models as shown in Table With large backbonc and multi-scale testing, our method further achieves fate-of-the-art accuracy 80.63% %

---

Table 7: Performance evaluation of KLD on classic horizontal detection.
Detector
Regression Loss
AP
APsa
APTs
AP.
AP..
m
AP;
Smooth LI
37.2
56.6
39.7
21.4
41.1
48.0
RetinaNet 10
GloU
37.4
56.7
39.7
22.2
41.7
48.1
KLD
38.0
56.4
40.6
23.3
43.2
49.3
Smooth LI
37,9
58.8
41.0
22.4
41.4
49.1
Faster RCNN
g
GloU
38,3
58.7
41.5
22.5
41.7
49.7
KLD
38.2
58.7
41.7
22.6
41.8
49.3
loU
36.6
56.0
38.8
21.0
40.6
47.0
FCOS
2
KLD
36.8
56.3
39.1
21.7
40.8
47.5
Table 8: AP on different objects on DOTA-vI O.. Here R-101 denotes 101 (likewise for R-50, R-152), and RX-101 and H-104 represent ResNext101 [49] and Hourglass- 104 (50), respectively. MS indicates that multi-scale training/testing is used. Red and blue indicate the top two performances.
MeBhed
Baskhoce
MS
PL.
BD
BR
G1F
5V
LV
SH
TC
BC
ST
SBF
R.A
HA
SF
HC
APp
[CN
K:101
76.30
47.7 70
10:30
64.43
87 30
10.00
30.31
19.10
16.30
$5.00
62.90
07.00
04:30
$1.31
08:25
79.53
SCRDes
68.36
68.24
Gliding Norea ST
R:101
64
85.00
13.01
73.
94.74
79.00
56.81
59.55
70.9
72.94
70.86
75
Madk Onn [
R00-01
19.50
05.05
54.71
72.M
56.53
76.10 16
85.63
59.85
83.61
46.0 4.
56.00
69.4
73.94
69.06
63.37
75.31
ComeeMappO00 Olb
R:101
69.83
$4.43
$4.00
70.25
77.06
78.32
$7.19
20.06
$4.89
55.27
56.46
69.23
74.13
71.56
66.06
76.03
FPN-CR
08.24
05.53
44
37.0
20.84
16.15
48.04
73.83
10
16.11
SCRDee
71.11
57.00
71.29
Balba [ST
Rell-00
88.81
82
60.03
50.82
18.54
86.06
$8.51
$0.80
$8.77
47.05
68.63
66.90
79.36
79.71
74.67
$0.10
Plpu 1i
DLA:33 K0
50710
00.70
28.10
(0120
15.30
64.30
24.00
11:30
10.40
10.30
31.10
57.10
01.90
04.00
(4130
0P-Diee Di--hhel [D]
H-:00
89.31
62.14
47.33
61.21
71.32
74.00
78,62
90.3
$2.23
$1.3 16.56
60.00
60.17
58.21
66.96
61.00
71.04
DAL
R-101
01
79. 6/
46.3
70.37
00.99
76 10
78.53
29.94
79.96
76 61
59.71
62:00
69:25
71.32
Singoodage
ES
77.83
75.79
78.50 06
20.50
91:71
57.00
69.77
65.13
72.30
79.43
53.58
65.22
DRN [+]]
16-148
82. 3.54
66:10
34.61
50.57
36.18
44 88
57.60
61.95
69 10
69.53
15.48
15.21
Poladgg K0
R.101
59.65
$7.07
48.14
70.97
78.53
50.34
$7.45
50.76
55.63
56.57
61.64
70.32
71.92
13,09
67.15
76.64
RDD M1]
R:101
59:15
$3.92
12.51
73.06
77.81
79.00
57.00
20.02
56.72
57.15
63.06
70.29
76.98
75.79
72.15
17:75
DWit
70
$4.51
06
83.71
76.05
71.51
18.30
75.95
79.33
UPC
10:-1
11.41
10.35
7k.11
30130
33.6
07.32
03.00
3010
R-Tba DS
R-152
00
$3.77
48.11
66.77
78.76
83.27
$7.84
30.82
55.38
$5.51
65.67
62.68
67.53
76.56
72.62
76.47
DAI: [k]
82:00
09.09
83:11
15.03
11
00
51 14
18.00 06
90.30
$4.97
$7.80
64.41
65.05
76.M6
72.19
04.35
T6
DCL 220
R-192
19:20
33.00
55.54
72.76
74.04
$2.50
37:51
90.67
36.59
56
67.00
26.88
73.29
70.56
69:00
17:37
RIDet
59.31
80.77
54.07
79.81
$1.99
$3.59
57:22
64.42
78.08
78:17
65.07
RPTha-CWWD
10:50
$6.17
55.00
69.35
18.72
84
$7.00
59.75
43.22
95.75
66.74
61.00
76.62
78.49
70.89
17:35
RP/Der KLD
R:50
59.90
$4.91
59.21
78.74
78.92
$3.95
57.41
59.39
56.63
56,09
71447
70.7
76.96
79.40
78.62
51.17
R-152
02
55.13
59.10 19
$1.33
78.82
84.3
$7.50
$9.90
$7.53
$7.00
72:57
71.35
77.12
79.34
78.68
$0.63
5 Discussions
Limitations. Despite the theoretical grounds and the promising experimental justifications. our method has an obvious limitation that it cannot be directly applied to quadrilateral detection
Potential negative societal impacts. Our findings provides a simple regression loss for high- precision rotation detection. However, our research may be applied to some sensitive fields, such as remote sensing, aviation, and unmanned aerial vehicles.
Conclusion. Departure from the vast existing literature in object detection, in this paper we have designed a new regression loss for rotation detection from scratch and consider the popular horizontal detection its special case. Specifically, we calculate the KLD between the Gaussian distributions corresponding to the rotated bounding box as the regression loss, and we find that in the learning procedure guided by the KLD loss, the gradient of the parameters according to the characteristics of the object which is a desirable property for robust object detection, regardless its rotation, size and aspect ratio etc. We also proved that KLD has scale invariance, which is crucial for detection tasks. Interestingly, we have shown that KLD can be degenerated into the currently commonly used nnrm loss in the horizontal detection task. Extensive experimental results across different detectors and datasets show the effectiveness of our approach.
be dynamically adjusted
Acknowledgments and Disclosure of Funding
This work was partly supported by National Key Research and Development Program of China (2018AAA0100704), Shanghai Municipal Science and Technology Major Project (201ISHZDZX0102) and NSFC (U20B2068, 72061 127003, 61972250). Xue Yang was also partly supported by Wu Wen Jun Honorary Doctoral Scholarship, Al Institute, Shanghai Jiao Tong Univer- sity, Shanghai, China.
10

---

References
[1] X. Yang, H. Sun. K. Fu, J. Yang, X. Sun, M. Yan, and Z. Guo, "Automatic ship detection in remote sensing images from google earth of complex scenes based on multiscale rotation dense feature pyramid networks," " Remote Sensing, vol. 10, no. 1, p. 132, 2018.
[2] X. Yang, H. Sun, X. Sun, M. Yan, Z. Guo, and K. Fu, "Position detection and direction prediction for -rrienee ships via multitask rotation region convolutional neural network," IEEE Access, vol. 6, pp. 50 839-50 849, 2018.
[3] X. Yang, J. Yang. J. Yan, Y. Zhang. T. Zhang, Z. Guo, X. Sun, and K. Fu, "Serdet: Towards more robust detection for small, cluttered and rotated objects," Proceedings of the IEEE International Conference on Computer Vision, 2019. pp. 8232-8241.
[4] X. Yang and J. Yan, "Arbitrary-oriented object detection with circular smooth label," in European Confer- ence on Computer Vision. Springer, 2020, pp. 677-694.
[5] X. Yang, J. Yan, M. Qi. W. Wang, Z. Xiaopeng, and T. Qi, "Rethinking rotated object detection with gaussian wasserstein distance loss," in International Conference on Machine Learning, 2021.
[6] Q. Ming, L. Mino, Z. Zhou, J. Song. and X. Yang. "Sparse label assignment for oriented object detection in acrial images." Remote Sensing, vol. 13, no. 14, 2664, 2021.
[7] R. Girshick. "Fast r-cnn," in Proceedings of the IEEE International Conference Computer Vision 2015, pp. 1440-1448.
[8] S. Ren, K. He, R. Girshick, and Sun, "Faster r-cnn: Towards real-time object detection with region proposal networks," in Advances in neural information processing systems, 2015, pp. 91-99.
19] T.-Y. Lin, P. Dollár, R. Girshick, He, B. Hariharan, and S. Belongie, "Feature pyramid networks for object detection," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 2117-2125.
[10] Lin, P. Goyal, R. Girshick, K. He, and P. Dollár, "Focal loss for dense object detection," in Proceedings of the IEEE International Conference Computer Vision, 2017, pp. 2980-2988.
[11] J. Dai, Y. Li, K. He, and Sun, "R-fen: Object detection via region-based fully convolutional networks." in Advances in neural information processing systemx, 2016, pp. 379-387.
[12] Ding, N. Xue, Long, G.-S. Xia, and Q. Lu, "Learning roi transformer for oriented object detection in aerial images," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. 22588
[13] J. Han, J. Ding, J. Li, and G.-S. Xia, "Align deep features for oriented object detection." IEEE Transactions Geoscience and Remote Sensing, 2021.
[14] X. Pan, Y. Ren, K. Sheng. W. Dong. H. Yuan, X. Guo, C. Ma. and C. Xu, "Dynamic refinement network for oriented and densely packed object detection," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2020, pp. 11 207-11 216.
[15] Q Ming, Z. Zhou, L. Miao, H. Zhang, and L. Li, "Dynamic anchor learning for arbitrary-oriented object detection," in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, no. 3, 2021, pp. 2355-2363.
[16] S. Kullback and R. A. Leihler, "On information and sufficiency." The annals of mathematical statistics, vol. 22, no. 1. pp. 79-86, 1951.
[17] C. Villani, Optimal transport: old and Springer Science & Business Media, 2008, vol. 338.
[18] X. Yang, Y. Zhou, and J. Yan, "Alpharotate: A rotation detection benchmark using tensorflow," arXiv preprint arXiv:2111.666 2021.
[19] Redmon, Divvala, R. Girshick, and A. Farhadi, "You only look once: Unified, real-time object detection," in Proceedings of the IEEE conference on computer vision and pultern recognition, 2016, pp. 779-788.
[20] W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, and A. C. Berg, "Ssd: Single shot multibox detector"" in European conference computer vision. Springer, 2016, pp. 21-37.

---

[21] Z. Tian, C. Shen, H. Chen, and T. He. "Feos: Fully convolutional one-stage object detection," in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 9627-9666.
[22] X. Zhou, D. Wang, and Krähenbühl, "Objects as points," arXiv preprint arXiv:1904.07850, 2019.
[23] Z. Yang, S. Liu, H. Hu, L. Wang, and S. Lin, "Reppoints: Point set representation for object detection,"" in P Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 9657 9666.
[24] N. Carion, F. Massa, G. Synnaeve, N. Usunier, A. Kirillov, and S. Zagoruyko, "End-to-end object detection with transformers." " in Earopean Conference on Computer Vision. Springer, 2020, pp. 213-229.
[25] X. Zhu, W. Su, L. Lu, B. Li, X. Wang, and J. Dai, "Deformable detr: Deformable transformers for end-lo-end object detection," in International Conference Learning Representations, 2020.
[26] H. Rezatofighi, N. Tsoi, Gwak, A. Sadeghian, Reid, and S. Savarese, "Generalized intersection over union: A metric and a loss for bounding box regression," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. 358-666.
[27] Z. Zheng, P. Wang. W. Liu, Li. R. Ye, and D. Ren, "Distance-iou loss: Faster and better learning for bounding box regression." in Proceedings of the AAAI Conference Artificial Intelligence. 2020, pp. 12 1300-1300..
[28] X. Yang. J. Yan, Z. Feng, and T. He, "R3det: Refined single-stage detector with feature refinement for rotating object," in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, no. 4, 2021, pp. 3163 3171.
[29] Y. Jiang. X. Zhu, Wang, S. Yang, Li, H. Wang, Fu, and Z. Luo, "R2cmm: rotational region enn for orientation robust scene text detection," " arXiv preprint arXiv: I 170600957, 2017.
[30] J. Ma, W. Shao, H. Ye, L. Wang, H. Wang, Y. Zheng, and X. Xue, "Arbitrary-oricntee seene text detection via rotation proposals," IEEE Transactions Multimedia, vol. 20, no. 11, pp. 3111-3122, 2018.
[31] S. Azimi, E. Vig, Bahmanyar, M. Kömer, and P. Reinartz, "Towards multi-class object detection in unconstrained remote sensing imagery," in Asian Conference on Computer Vision. Springer, 2018, pp. 150-165.
[32] 7. Chen, K. Chen, W. Lin, J. See, H. Yu, Y. Ke, and C. Yang, "Piou loss: Towards accurate oriented object detection in complex environments," in European Conference on Computer Vision. Springer, 2020, pp- 195-211.
[33] Q. Ming, L. Miao, Z. Zhou, and Y. Dong, "Cíc-net: A critical feature capturing network for arbitrary- oriented object detection in remote sensing images," arXiv preprint arXiv:2101.0682 2021.
[34] Q. Ming, L. Miao, Z. Zhou, X. Yang, and Y. Dong, "Optimization for arbitrary-oriented object detection via representation invariance loss," IEEE Geoscience and Remote Sensing Letters, 2021.
[35] C. Manning and H. Schutze, Foundations of statistical natural language processing.
MIT press, 1999.
[36] H. Jeffreys, "An invariant form for the prior probability in estimation problems," Proceedings of the Royal Society of London. Series A. Mathematical and Physical Sciences, vol. 186, no. 1007, pp. 453-461, 1946.
[37] G.-S. Xia, X. Bai, Ding, Z. Zhu, S. Belongie, J. Luo, M. Datcu, Pelillo, and L. Zhang, "Dota: A large-scale dataset for object detection in aerial images," in Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition, 2018, pp. 3974-3983.
[38] H. Zhu, X. Chen, W. Dai, K. Fu, Q. Ye, and Jiao, "Orientation robust object detection in acrial images using deep convolutional neural network," in 2015 IEEE International Conference on Image Processing. IEEE, 2015, pp. 3735-3739.
[39] Z. Liu, L. Yuan, L. Weng, and Y. Yang, "A high resolution optical satellite image dataset for ship recog- nition and some new baselines." " in Proceedings of the International Conference Pattern Recognition Applications and Methods, vol. 2, 2017, pp. 324-331.
[40] D. Karatzas, L. Gomez Bigorda, A. Nicolaou, S. Ghosh. A. Bagdanov, M. Iwamura, J. Matas, L. Neumann, V. R. Chandrasekhar. S. Lu et al. "Iedar 2015 competition on robust reading." in 2015 13th International Conference on Document Analysis and Recognition. IEEE, 2015, pp. 1156-1160.
12

---

[41] N. Nayef, F. Yin, I. Bizid, H. Choi, Y. Feng, D. Karatzas. Z. Luo. U. Pal. C. Rigaud, J. Chazalon et al., "1cdar2017 robust reading challenge multi-lingual scene text detection and script identfiiation-rre-nlt in 2017 14th IAPR International Conference on Document Analysis and Recognition, vol. 1. IEEE, 2017,
pp. 1454-1459,
[42] C. Yao, X. Bai, W. Liu, Y. Ma, and 7. Tu, "Detecting texts of arbitrary orientations natural images," in Proceedings of the IEFE Conference on Computer Vision and Pattern Recognition. IEEF, 2012, pp- 1083-1090.
[43] M. Abadi, P. Barham, J. Chen, Z. Chen, A. Davis, J. Dean, M. Devin, S. Ghemawat, G. Irving, M. Isard et al., "Tensorflow: A system for large-scale scale machine learning," in 12th {USENIX) symposium operating systems design and implementation ((OSDf) 16), 2016, pp. 265-283.
[44] K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition." in Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition, 2016, pp. 770-778.
[45] Qian, X. Yang, S. Peng. J. Yan, and Guo, "Learning modulated loss for rotated object detection," in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, no. 3, 2021, pp. 4458-2466.
[46] X. Yang. L. Hou, Y. Zhou, W. Wang, and Yan. "Dense label encoding for boundary discontinuity free rotation detection," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2021, pp. 15 819-15 829.
[47] X. Yang. J. Yan, and T. He, "On the arbitrary-oriented object detection: Classification based approaches revisited," arXiv preprint 20t.:0000005597 2020.
[48] T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick, "Microsoft coco: Common objects in context," in European conference on computer vision. Springer, 2014, pp. 750-755.
[49] S. Xie, R. Girshick, P. Dollár, Z. Tu, and K. He, "Aggregated residual transformations for deep neural networks," " in Procedingss of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 1492-1500.
[50] A. Newell, K. Yang, and J. Deng. "Stacked hourglass networks for human pose estimation," in European Conference on Computer Vision. Springer, 2016, pp. 483 499.
[51] Y. Xu, M. Fu, Q. Wang, Y. Wang, K. Chen, G.-S. Xia, and X. Bai, "Gliding vertex on the horizontal bounding box for multi-oriented object detection," IEEE transactions on paltern analysis and machine intelligence, vol. 43, no. pp. 1452-1459, 2020.
[52] J. Wang. Ding, H. Guo. W. Cheng, T. Pan, and W. Yang. "Mask obb: A semantic attention drase mask oriented bounding box representation for multi-category object detection in aerial images," Remote Sensing, vol. 11, no. 24, p. 2930, 2019.
[53] Wang, W. Yang. H.-C. Li, H. Zhang, and G.-S. Xia, "Learning center probability map for detecting objects in aerial images," IEEE Transactions on Geoscience and Remote Sensing, vol. 59, no. 5, pp. 4307-4323, 2020.
[54] X. Yang, J. Yan, X. Yang, J. Tang, W. Liao, and T. He, "Serdet++: Detecting small, cluttered and rotated objects via instance-level feature denoising and rotation loss smoothing." arXiv preprint 200: 13316, 2020.
[55] J. Han, J. Ding, N. Xue. and G.-S. Xia, "Redet A rotation- equivariaativiiiinnnnn detector for aerial object detection." in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2021, pp. 2786 2795.
[56] F. Yu, D. Wang, E. Shelhamer, and T. Darrell, "Deep layer aggregation," " in Proceedings of the IEFE Conference on Computer Vision and Pattern Recognition, 2018. pp. 2403-2412.
[57] H. Wei, Y. Zhang, Chang, ISPRS Journal of Photogrammetry and Remote Sensing. vol. 169, pp. 268-279, 2020.
H. Wang, and X. Sun, "Oriented objects as pairs middle lines."
[58] L. Zhou, H. Wei, H. Li, W. Zhao, Y. Zhang, and Y. Zhang, "Arbitrary-oriented object detection remote sensing images based on polar coordinates." IEEE Access, vol. 8, pp. 223 373-223 384, 2020.
[59] J. Yi. Wu, B. Liu, Q Huang. H. Qu, and D. Metaxas, "Oriented object detection in aerial images with box boundary aware vectors." in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2021, pp. 2150 2159..
13

---

[60] P. Zhao, Z. Qu. Y. Bu. W. Tan, and Q. Guan, "Polardet: A fast, more precise detector for rotated target in aerial images." " International Journal of Remote Sensing, vol. 42, 15, pp. 5821-5851, 2021.
[61] B. Zhong and K. Ao, "Single-stage rotation-decoupled detector for oriented object," Remote Sensing, vol. 12, no. 19, p. 3262, 2020.
14


Reviews:
[{"review_id": "wmt5dUBtsiq", "reviewer": "Reviewer_7RYn", "summary": "The paper proposes an algorithm that can detect slender objects,\nlike text in images or boats in aerial images. The work mainly focuses on\nthe bounding box regression task and includes a parameter for box rotation.\nBounding boxes are represented as Gaussian functions instead of axis-aligned\nboxes. The scale invariant KL-divergence loss is applied to the regression task.\nAn analysis of the loss function proclaims that the computation of gradients,\nduring optimization, are adjusted with respect to the bounding box parameters.", "questions": "", "limitations": "", "rating": 2, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper is not reproducible, motivations and explanations\nare incomplete. This paper is not yet ready for NeurIPS 2021.\n\n1. The paper gives a vague description of the problem, why is it important\nand what are the challenges. There is insufficient motivation. A clear\ndescription of why and the final end-use should be included. In this end-use, why are rotated bounding boxes better compared to another high-\nprecision object representation, for example segmentation.\n2. A clear motivation of why it is more correct to directly predict bounding\nbox parameters, rather then parameter offsets, is not given. Both can be\nequally good depending on prior assumptions.\n3. Line 229, what kind of data augmentation?\n4. It is a strong claim to say that horizontal bounding boxes are a special\ncase of something else without any references. Equations (4) and (16) are\nnot very similar. There are large differences, more terms and also different\nnorms.\n5. Missing references. Claims: lines 102, 104-105, 141-144, 212. Terms: lines\n275, 276, 279.\n6. Which methods do you refer to when you say they are not scale invariant?\nIn SSD, the offsets are very near a standard normal distribution. It is\nunclear why a scale invariant loss versus a simple scaling of the loss is\nbeneficial.\n7. You want to convey that the KL-loss dynamically adjust gradients w.r.t\nwidth and height. However, the details do\nnot clearly explain this statement. E.g. is function f (13) defined here or explained somewhere else?\n8. Usually L1-norm is better when there are outliers. Here outliers are allowed to contribute more to the loss, due to the quadratic terms in the KL-divergence loss. From the ablation this is solved by (18). This extra normalization questions if the KL-divergence is actually contributing\nor simply produces noise in the results. This loss would have been better motivated if Table 6 was extended with more of the state-of-the-art\ndetectors, where axis aligned boxes are used.\n9. References [6,7,8,9,10] do not include an extra log term in the bounding\nbox regression. In (4) and extra log term is added.\n10. Line 63, not well motivated or explained.\n11. Line 191, is f the same as in (13), (14), (15)?\n12. Needs proof reading. E.g. line 38, (13), line 143, line 194, line 279.\n\nUpdate after rebuttal:\nseveral questions have been addressed in the rebuttal. However, question 8 is not sufficiently accurately addressed. Also: what does \"We will discuss above in detail in the final version.\" mean concretely.\nGiven the responses from the reviewers, I have raised my assessment, but the paper is still not of sufficient quality for acceptance.\n\nUpdate after discussion with the authors:\nThe disrespectful tone during the discussion is unprofessional and not suitable for a scientific top tier conference. The rating is changed accordingly.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "It is not clear why the listed fields (line\n288) are sensitive. An example for one of the fields could give a more complete\nunderstanding of the negative impacts.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "18"}, "extra_text": {}}, {"review_id": "g7jgQn4DNPm", "reviewer": "Reviewer_eGtr", "summary": "This paper focuses on learning high precision bounding boxes for rotated object detections. Instead of directly predicting the size of the box, they turn the box into a gaussian distribution and use the KL divergence between the gaussian distributions as the regression loss. They show that KLD is modulated where the optimization of one parameter will be affected by other hyper-parameters. For example, the gradients for the orientation involve the width and height of a box. Experiments show that the proposed loss function significantly improve the performance of a detector on different aerial image datasets and scene text datasets.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The main contribution of this paper is that it shows why KLD is a better option than existing losses to train a network for rotated objects both analytically and empirically. Their results also generalize well across different datasets. Although this paper does not propose any new loss or new formulation of a task, this shows a strong result for future works on rotated object detection. Furthermore, this paper does a good job on explaining everything very clearly. It is very easy to follow. Overall, I think this is a good paper.\n\nOne downside of this approach is that it does not have much advantage on horizontal object detection such COCO as shown in table 6, which limits its practicality, considering that some corner-based detectors [1] already perform well at high IoU level on COCO. Having said that, I do not consider this is a major issue as this works very well rotated object detection which is also valuable.\n\nThe authors show that they are able to achieve a higher performance with a smaller network. So I would suggest the authors to include the inference time of their methods and compare it with other works.\n\n1. Hei Law, Jia Deng: CornerNet: Detecting Objects as Paired Keypoints. Int. J. Comput. Vis. 128(3): 642-656 (2020)", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "8rmUCSy33uZ", "reviewer": "Reviewer_MV1m", "summary": "This paper presents a new design of the regression loss function for oriented object detection. When modeling the rotated bounding boxes as 2D Gaussian distributions, the authors introduced the Kullback-Leibler Divergence for oriented bounding boxes regression as an alternative to the commonly-used Smooth-l1 loss function. The experimental results show that the proposed loss function could obtain better performance on several benchmarks. ", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "- This paper extended the idea of modeling the oriented bounding boxes as Gaussian distributions [1]. The main difference between this paper and [1] is using the KL Divergence instead of the Gaussian Wasserstein Distance as the regression loss function for the learning of oriented bounding boxes. Although better performance could be obtained, this paper did not present insights for the reason why using KL Divergence.  Accordingly, this paper is actually another attempt for finding a better-performing distance function under the framework of Gaussian modeling of oriented object detection. Without further exploiting the insights behind the proposed KLD loss, the main contributions claimed about the novelty of the proposed loss function by this paper are not that significant.\n\n- The authors discussed the **self-modulated mechanism** of the KLD loss, however, it is only a trivial observation. As the final evaluation metric is based on the IoU computation between two bounding boxes in the 2D Euclidean space, the more strict requirement for the objects with a larger aspect ratio should be taken into account for the sake of better IoU. A feasible solution is improving the weight for learning large aspect-ratio objects. \n\n- From another perspective, the proposed KLD loss couples the orientation and the width-height estimation together. As shown in equation (10), the residual in $x$ and $y$ directions are rotated by a rotation matrix $R(\\theta) \\in \\text{SO}(2)$, which is actually a conversion from the polar coordinate system to the Euclidean one.\n\n- The KLD loss only addressed the inaccurate angular regression of the **oriented** object detection for the larger aspect-ratio objects. For the detection of the horizontal bounding boxes, the problem of inaccurate regression for the width and height still remains.\n\n## References\n[1] X. Yang, J. Yan, M. Qi, W. Wang, Z. Xiaopeng, and T. Qi, “Rethinking rotated object detection with Gaussian Wasserstein distance loss,” in International Conference on Machine Learning, 2021.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors have adequately addressed the limitations and potential negative societal impact of their work.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The reviewers have discussed the paper and have not come to an agreement.
Reasons to accept: the proposed approach is noted to be simple and yet effective, and it can improve the performance of various existing object detectors on different datasets significantly. Most of the issues raised in the reviews were properly addressed in the author responses (including extra experimental results).
Reasons to reject: some of the reviewers were not satisfied with the provided responses and decided not to support the paper.

I think that the overall pros of the paper outweigh the cons and would like to recommend accepting the paper.

Decision: Accept (Poster)