Learning High-Precision Bounding Box for Rotated Object Detection via Kullback-Leibler Divergence
Year: 2021

Abstract:
Existing rotated object detectors are mostly inherited from the horizontal detection paradigm, as the latter has evolved into a well-developed area. However, these detectors are difficult to perform prominently in high-precision detection due to the limitation of current regression loss design, especially for objects with large aspect ratios. Taking the perspective that horizontal detection is a special case for rotated object detection, in this paper, we are motivated to change the design of rotation regression loss from induction paradigm to deduction methodology, in terms of the relation between rotation and horizontal detection. We show that one essential challenge is how to modulate the coupled parameters in the rotation regression loss, as such the estimated parameters can influence to each other during the dynamic joint optimization, in an adaptive and synergetic way. Specifically, we first convert the rotated bounding box into a 2-D Gaussian distribution, and then calculate the Kullback-Leibler Divergence (KLD) between the Gaussian distributions as the regression loss. By analyzing the gradient of each parameter, we show that KLD (and its derivatives) can dynamically adjust the parameter gradients according to the characteristics of the object. For instance, it will adjust the importance (gradient weight) of the angle parameter according to the aspect ratio. This mechanism can be vital for high-precision detection as a slight angle error would cause a serious accuracy drop for large aspect ratios objects. More importantly, we have proved that KLD is scale invariant. We further show that the KLD loss can be degenerated into the popular ln-norm loss for horizontal detection. Experimental results on seven datasets using different detectors show its consistent superiority, and codes are available at https://github.com/yangxue0827/RotationDetection.

heading: 1 Introduction
text: As a fundamental building block for visual analysis across aerial images, scene text etc., rotated object detection has recently been developed rapidly [1, 2, 3, 4, 5, 6], which benefit themselves from the well-established horizontal detection approaches [7, 8, 9, 10, 11]. Specifically, many works [12, 13, 14, 15] build themselves upon the previously established horizontal detection pipeline from an inductive perspective, as shown in Figure 1(a). However, these detectors are often unable to cope with challenging scenes well due to the limitations of current regression loss, such as large aspect ratio objects, dense scenes, etc., resulting in obvious disadvantages in high-precision detection. ∗Part of the work was done during an internship at Huawei Inc. †Correspondence author is Junchi Yan.
35th Conference on Neural Information Processing Systems (NeurIPS 2021).
(a) Previous methods follow the induction paradigm from special horizontal to general rotated detection. (b) Our proposed method adopts a deduction methodology from general rotated to special horizontal detection.
Figure 1: Methodological road-map difference between horizontal detection (special case) and rotation detection (general case) in the previous methods [1, 12, 13, 14, 15] and the proposed method.
In this paper, we take a step back, and aim to develop (from a deductive perspective) a unified regression framework for rotation detection and its special case: horizontal detection. In fact, our new framework enjoys a coherent property that it can be degenerated into the current commonly used regression loss (e.g. ln-norm) in special cases (horizontal detection), as shown in Figure 1(b).
For a devising a rotation regression loss for high-precision rotation detection, one important observation is that the importance of different parameters to different types of objects can vary. For example, the angle parameter (θ) and the center point parameter (x, y) are important for large aspect ratio objects and small objects, respectively. In another word, it is conjectured that regression loss should be self-modulated during the learning process and calls for more dynamic optimization strategy.
Inspired by the above ideas, we first convert the rotated bounding box B(x, y, h, w, θ) into a 2-D Gaussian distribution N (µ,Σ). As a standard distance metric, we then use the Kullback-Leibler Divergence (KLD) [16] to calculate the distribution distance between the predicted bounding box and ground truth as the regression loss. We compare KLD with Smooth L1 loss [7] and another distance metric, Gaussian Wasserstein Distance (GWD) [5, 17], and find that KLD has a more complete parameter optimization mechanism. In particular, by analyzing the gradient of the parameters during learning, we show that the optimization of one parameter will be affected by other parameters (as the gradient weight). It means that the model will adaptively adjust the optimization strategy given a specific configuration of an object for detection, as shown can lead to excellent performance in high-precision detection. In addition, KLD is proven scale invariant, which is an important property that Smooth L1 loss and GWD do not possess. As the horizontal bounding box is a special case of the rotated bounding box, we show that KLD can also be degenerated into the ln-norm loss as commonly used in existing horizontal detection pipeline. The highlights of this paper are four-folds:
1) Differing from the dominant existing practices that build rotation detectors heavily upon the horizontal detectors, we develop new rotation detection loss from scratch and show that it is coherent with existing horizontal detection protocol in its degenerated case for horizontal detection.
2) To achieve a more principled measurement between the prediction and ground truth, instead of computing the difference for each physically-meaningful parameter related to the bounding box which are in different scales and units, we innovatively convert the regression loss of rotation detection into the KLD of two 2-D Gaussian distributions, leading to a clean and coherent regression loss.
3) Through the gradient analysis of each parameter in KLD, we further find that the self-modulated optimization mechanism of KLD greatly promotes the improvement of high-precision detection, which verify the advantage of our loss design. More importantly, we have theoretically shown (in appendix) that KLD is scale invariant for detection, which is crucial for the rotation cases.
4) Extensive experimental results on seven public datasets and two popular detectors show the effectiveness of our approach, which achieves new state-of-the-art performance for rotation detection. The source codes [18] are made public available.

heading: 2 Background
text: We first generally discuss the related works on both horizontal and rotated object detection. Then we summarize the current design paradigm of rotation regression loss from two kinds of methodologies,
as shown in Figure 1: one is inductive that tries to develop the general rotation detection from the special and classic horizontal detection pipeline. While the other is deductive that aims to devise a general rotation detection pipeline with horizontal detection as its special case.

heading: 2.1 Related Works
text: Horizontal object detection. Horizontal object detection which covers most existing detection literature, normally uses a horizontal bounding box to represent the object. The mainstream classical object detection algorithms can be roughly divided according to the following standards: Two[7, 8, 9, 11] or Single-stage [10, 19, 20] object detection, Anchor-free [21, 22, 23] or Anchor-based [8, 9, 10] object detection and CNN [8, 10, 21] or Transformer-based [24, 25] object detection. Although the pipelines may vary, the mainstream regression loss often uses the popular ln-norm loss (such as smooth L1 loss) or IoU-based loss (such as GIoU [26], and DIoU [27]). These abovementioned detectors have also been widely used in other scenarios and have achieved satisfactory performance. However, horizontal detectors do not provide accurate orientation and scale information.
Rotated object detection. Recent advances in rotation detection [3, 4, 12, 14, 28] are mainly driven by adapting the horizontal object detectors with rotated bounding boxes to represent multi-oriented objects. To accurately predict the rotated bounding box, most rotation detection methods extend the ln-norm [12, 15, 29, 30, 31] used in horizontal detection, or construct a differentiable approximate IoU loss [3, 5, 32]. From scratch, we try to change the design of rotation regression loss from induction paradigm to deduction methodology, which in fact is a generalization to the horizontal case.
In the following, we describe the existing works from the induction and deduction methodologies.

heading: 2.2 Inductive Thinking of Loss Design: from Special Horizon to General Rotation Detection
text: Regression loss is a vital part of most current object detection algorithms. For horizontal bounding box regression, the model [7, 8, 9, 10, 11] mainly outputs four items for location and size:
tpx = xp − xa wa , tpy = yp − ya ha , tpw = ln ( wp wa ) , tph = ln ( hp ha ) (1)
to match the four targets from the ground truth
ttx = xt − xa wa , tty = yt − ya ha , ttw = ln ( wt wa ) , tth = ln ( ht ha ) (2)
where x, y, h, w denote the center coordinates, height and width, respectively. Variables xt, xa, xp are for the ground-truth box, anchor box, and predicted box, respectively (likewise for y, w, h).
Extending the above horizontal case, existing rotation detection models [1, 12, 13, 14, 15] also use regression loss which simply involves an extra angle parameter θ:
tpθ = f(θp − θa), t t θ = f(θt − θa) (3)
where f(·) is used to deal with angular periodicity, such as trigonometric functions, modulo, etc. The overall regression loss for rotation detection is:
Lreg = ln-norm (∆tx,∆ty,∆tw,∆th,∆tθ) (4)
where ∆tx = tpx− ttx = ∆xwa , ∆ty = t p y− tty = ∆y ha , ∆tw = tpw− ttw = ln(wp/wt), ∆th = t p h− tth = ln(hp/ht), and ∆tθ = t p θ − ttθ = ∆θ.
It can be seen that parameters are optimized independently, making the loss (or detection accuracy) sensitive to the under-fitting of any of the parameters. This mechanism is fatal to high-precision detection. Taking the left side of Figure 2 as an example, the detection result based on the Smooth L1 loss often shows the deviation of the center point or angle. Moreover, different types of objects have different sensitivity to these five parameters. For example, the angle parameter is very important for detecting objects with large aspect ratios. This requires to select an appropriate set of weights given a specific single object sample during the training, which is nontrivial or even unrealistic.

heading: 2.3 Deductive Thinking of Loss Design: from General Rotation to Special Horizon Detection
text: To break the original inductive design paradigm, we adopt deductive paradigm to construct more accurate rotation regression loss. Here we rephrase the main idea in the recent work [5], which converts a arbitrary-oriented bounding box B(x, y, h, w, θ) into a 2-D Gaussian N (µ,Σ), as illustracted in Figure 3. Then the distance between two Gaussian is calculated as the final loss. Specifically, the conversion is:
µ =(x, y)>
Σ1/2 =RΛR> = ( cos θ − sin θ sin θ cos θ )( w 2 0 0 h
2
)(
cos θ sin θ − sin θ cos θ
)
= ( w 2 cos2 θ + h 2 sin2 θ w−h 2
cos θ sin θ w−h 2 cos θ sin θ w 2 sin2 θ + h 2 cos2 θ ) (5)
where R represents the rotation matrix, and Λ represents the diagonal matrix of eigenvalues.
The recent work [5] analyzes that the introduction of N (µ,Σ) can solve the inconsistency between metric and loss, boundary discontinuity and square-like problem. On this basis, we further study how to design high-precision detection regression loss through new parameter space. Our view is that the self-modulated mechanism is positively correlated with the final high-precision performance.
Gaussian Wasserstein Distance. The Wasserstein distance [5, 17] between two probability measures Xp ∼ Np(µp,Σp) and Xt ∼ Nt(µt,Σt) expressed as:
Dw(Np,Nt)2 = ‖µp − µt‖22︸ ︷︷ ︸ center distance + Tr(Σp + Σt − 2(Σ1/2p ΣtΣ1/2p )1/2)︸ ︷︷ ︸ coupling terms about hp ,wp and θp
(6)
Eq. 6 shows that the Gaussian Wasserstein Distance (GWD) is mainly divided into two parts: the distance between the center points (x, y) and the coupling terms about h, w and θ. Accordingly, the regression loss based on GWD can be regarded as a semi-coupled loss. Although GWD can greatly improve the performance of high-precision rotation detection due to the coupling between part of the parameters, the independent optimization of the center point make the detection result slightly shifted (see Figure 2). Note that GWD is not scale invariant, which is not detection friendly.
When all the boxes are horizontal (θ = 0◦), Eq. 6 can be further simplified:
Dhw(Np,Nt)2 =‖µp − µt‖22 + ‖Σ1/2p −Σ 1/2 t ‖ 2 F =(xp − xt)2 + (yp − yt)2 + ( (wp − wt)2 + (hp − ht)2 ) /4
=l2-norm(∆x,∆y,∆w/2,∆h/2)
(7)
where ‖ · ‖F is the Frobenius norm. Although Eq. 7 can still be used as the regression loss of horizontal detection, Eq. 4 and 7 are not completely consistent.
Although GWD scheme has played a preliminary exploration of the deductive paradigm, it does not focus on achieving high-precision detection and scale invariance. In the following, we will propose our new approach based on the Kullback-Leibler divergence (KLD) [16].

heading: 3 Proposed Approach
text: Kullback-Leibler Divergence. To explore the more appropriate regression loss, we adopt the Kullback-Leibler divergence (KLD) [16]. Similarly, the KLD between two 2-D Gaussian is:
Dkl(Np||Nt) = 1 2 (µp − µt)>Σ−1t (µp − µt)︸ ︷︷ ︸
term about xp and yp
+ 1 2 Tr(Σ−1t Σp) + 1 2 ln |Σt| |Σp|︸ ︷︷ ︸
coupling terms about hp ,wp and θp
−1 (8)
or Dkl(Nt||Np) = 1
2 (µp − µt)>Σ−1p (µp − µt) +
1 2 Tr(Σ−1p Σt) + 1 2 ln |Σp| |Σt|︸ ︷︷ ︸
chain coupling of all parameters
−1 (9)
It can be seen that each item in Dkl(Nt||Np) is composed of partial parameter coupling, which makes all parameters form a chain coupling relationship. In the optimization process of the KLD-based detector, the parameters influence each other and are jointly optimized which make optimization mechanism of the model is self-modulated. In contrast, Dkl(Np||Nt) and GWD are both semicoupled, but Dkl(Np||Nt) has a better central point optimization mechanism. Although KLD is asymmetric, we find that the optimization principles of these two forms are similar by analyzing the gradients of various parameters and experimental results. Take the relatively simple Dkl(Np||Nt) as an example, according to Eq. 5, each item of Eq. 8 can be expressed as
(µp − µt)>Σ−1t (µp − µt) = 4 (∆x cos θt + ∆y sin θt)
2
w2t +
4 (∆y cos θt −∆x sin θt)2
h2t (10)
Tr(Σ−1t Σp) = h2p w2t sin2 ∆θ + w2p h2t sin2 ∆θ + h2p h2t cos2 ∆θ + w2p w2t cos2 ∆θ (11)
ln |Σt| |Σp| = ln h2t h2p + ln w2t w2p
(12)
where ∆x = xp − xt,∆y = yp − yt,∆θ = θp − θt. Analysis of high-precision detection. Without loss of generality, we set θt = 0◦, then
∂Dkl(µp)
∂µp =
( 4
w2t ∆x,
4
h2t ∆y
)> (13)
The weights 1/w2t and 1/h 2 t will make the model dynamically adjust the optimization of the object position according to the scale. For example, when the object scale is small or an edge is too short, the model will pay more attention to the optimization of the offset of the corresponding direction. For this kind of object, a slight deviation on the corresponding direction will often cause a sharp drop in IoU. When θt 6= 0◦, the gradient of the object offset (∆x and ∆y) will be dynamically adjusted according to the θt for better optimization. In contrast, the gradient of the center point in GWD and L2-norm are
∂Dw(µp) ∂µp = (2∆x, 2∆y)> and ∂L2(µp)∂µp = ( 2 w2a ∆x, 2h2a ∆y)>. The former cannot adjust
the dynamic gradient according to the length and width of the object. The latter is based on the length and width of the anchor (wa, ha) to adjust the gradient instead of the target object (wt, ht), which is almost ineffective for those detectors [3, 13, 15, 28, 29, 33, 34] that use horizontal anchors for rotation detection. More importantly, they are not related to the angle of the target object. Therefore, the detection result of the GWD-based and Ln-norm models will show a slight deviation, while the detection result of the KLD-based model is quite accurate, as shown in Figure 2.
For hp and wp, we have
∂Dkl(Σp) ∂ lnhp = h2p h2t cos2 ∆θ + h2p w2t sin2 ∆θ − 1, ∂Dkl(Σp) ∂ lnwp = w2p w2t cos2 ∆θ + w2p h2t sin2 ∆θ − 1 (14)
On the one hand, the optimization of the hp and wp is affected by the ∆θ. When ∆θ = 0◦, ∂Dkl(Σp) ∂ lnhp = h2p h2t − 1, ∂Dkl(Σp)∂ lnwp = w2p w2t − 1, which means that the smaller targeted height or width leads to heavier penalty on its matching loss. This is desirable, as smaller height or width needs higher matching precision. On the other hand, the optimization of ∆θ is also affected by hp and wp:
∂Dkl(Σp)
∂θp = ( h2p − w2p w2t + w2p − h2p h2t ) sin 2∆θ (15)
when wp = wt, hp = ht, then ∂Dkl(Σp)
∂θp = ( h2t w2t + w2t h2t − 2 ) sin 2∆θ ≥ sin 2∆θ, the condition for
the equality sign is ht = wt. This shows that the larger the aspect ratio of the object, the model will pay more attention to the optimization of the angle. This is the main reason why the KLD-based model has a huge advantage in high-precision detection indicators as a slight angle error would cause a serious accuracy drop for large aspect ratios objects. Through the above analysis, we find that when one of the parameters is optimized, the other parameters will be used as its weight to dynamically adjust the optimization rate. In other words, the optimization of parameters is no longer independent, that is, optimizing one parameter will also promote the optimization of other parameters. The optimization of this virtuous circle is the key to KLD as an excellent rotation regression loss. In addition, Dkl(Nt||Np) has similar properties, refer to appendix for details. Scale invariance. For a full-rank matrix M, |M| 6= 0, we have Dkl(Np||Nt) = Dkl(Np′ ||Nt′ ), where Xp′ = MXp ∼ Np(Mµp,MΣpM>), Xt′ = MXt ∼ Nt(Mµt,MΣtM>). Therefore, the affine invariance (including scale invariance when M = kI, where I denotes identity matrix) of KLD can be proven (see proof in appendix). Compared with Ln-norm and GWD, KLD is more suitable for replacing the non-differentiable rotated IoU loss for its consistency with detection metric.
Horizontal special case. For horizontal detection, combine Eq. 8 to Eq. 12, we have
Dhkl(Np||Nt) = 1
2 ( w2p w2t + h2p h2t + 4∆2x w2t + 4∆2y h2t + ln w2t w2p + ln h2t h2p − 2 )
=2l2-norm(∆tx,∆ty) + l1-norm(∆tw,∆th) + 1
2 l2-norm(
1
∆tw ,
1
∆th )− 1
(16)
where the first two terms of Eq. 16 are very similar to Eq. 4, and the divisor part of the two terms x and y is the main difference ( ∆xwt vs. ∆x wa ).
Variants of KLD. We have also introduced some variants [35, 36] of KLD to further verify the influence of asymmetry on rotation detection can be ignored. The variants mainly including
Dkl_min(max)(Np||Nt) = min(max) (Dkl(Np||Nt),Dkl(Nt||Np))
Djs(Np||Nt) = 1
2
( Dkl ( Nt|| Np +Nt
2
) + Dkl ( Np|| Np +Nt
2 )) Djef (Np||Nt) =Dkl(Nt||Np) + Dkl(Np||Nt)
(17)
Rotation regression loss. The whole training process of detector is as follows: i) predict offset (tpx, t p y, t p w, t p h, t p θ); ii) decode prediction box; iii) convert prediction box and target ground-truth into Gaussian distribution; iv) calculate KLD of two Gaussian distributions. Therefore, the inference time remains unchanged. We normalize the distance function as our final regression loss Lreg:
Lreg = 1− 1
τ + f(D) , τ ≥ 1 (18)
where f(·) denotes a non-linear function to transform the distance D to make the loss more smooth and expressive. In this paper, we mainly use two nonlinear functions, sqrt(D) and ln(D + 1). The hyperparameter τ modulates the entire loss. The multi-task loss is:
L = λ1 Npos Npos∑ n=1 Lreg(bn, gtn) + λ2 N N∑ n=1 Lcls(pn, tn) (19)
where Npos and N indicate the number of positive and all anchors. bn denotes the n-th bounding box, gtn is the n-th target ground-truth. tn denotes the label of n-th object, pn is the n-th probability distribution of various classes calculated by sigmoid function. The hyper-parameter λ1, λ2 control the trade-off and are set to {2, 1} by default. The classification loss Lcls is set as focal loss [10].

heading: 4 Experiment

heading: 4.1 Datasets and Implementation Details
text: Our experiments are conducted over a variety of datasets, including three large-scale public datasets for aerial images i.e. DOTA [37], UCAS-AOD [38], HRSC2016 [39], as well as scene text dataset ICDAR2015 [40], MLT [41] and MSRA-TD500 [42].
DOTA is one of the largest dataset for oriented object detection in aerial images with three released versions: DOTA-v1.0, DOTA-v1.5 and DOTA-v2.0. DOTA-v1.0 contains 15 common categories, 2,806 images and 188,282 instances. The proportions of the training set, validation set, and testing set in DOTA-v1.0 are 1/2, 1/6, and 1/3, respectively. In contrast, DOTA-v1.5 uses the same images as DOTA-v1.0, but extremely small instances (less than 10 pixels) are also annotated. Moreover, a new category, containing 402,089 instances in total is added in this version. While DOTA-v2.0 contains 18 common categories, 11,268 images and 1,793,658 instances. Compared to DOTA-v1.5, it further includes the new categories. The 11,268 images in DOTA-v2.0 are split into training, validation, test-dev, and test-challenge sets. We divide the images into 600× 600 subimages with an overlap of 150 pixels and scale it to 800× 800, in line with the cropping protocol in literature [5, 28]. UCAS-AOD contains 1,510 aerial images of approximately 659× 1, 280 pixels, with two categories of 14,596 instances in total. In line with [31, 37], we randomly select 1,110 for training and 400 for testing. HRSC2016 contains images from two scenarios including ships on sea and ships close inshore. The training, validation and test set include 436, 181 and 444 images.
ICDAR2015, MLT and MSRA-TD500 are commonly used for oriented scene text detection and spotting. ICDAR2015 includes 1,000 training images and 500 testing images. ICDAR2017 MLT is a multi-lingual text dataset, which includes 7,200 training images, 1,800 validation images and 9,000 testing images. MSRA-TD500 dataset consists of 300 training images and 200 testing images.
We use Tensorflow [43] to implement the proposed methods on a server with Tesla V100 and 32G memory. The experiments are all initialized by ResNet50 [44] by default unless otherwise specified. Weight decay and momentum are set 0.0001 and 0.9, respectively. We employ MomentumOptimizer over 8 GPUs with a total of 8 images per minibatch (1 image per GPU).
All the used datasets are trained by 20 epochs in total, and the learning rate is reduced tenfold at 12 epochs and 16 epochs, respectively. The initial learning rate is set to 5e-4. The number of image iterations per epoch for DOTA-v1.0, DOTA-v1.5, DOTA-v2.0, UCAS-AOD, HRSC2016, ICDAR2015, MLT and MSRA-TD500 are 54k, 64k, 80k, 5k, 10k, 10k, 10k and 5k respectively, and doubled if data augmentation (include random rotation, flipping, and graying) or multi-scale training is used.

heading: 4.2 Ablation Study and Further Comparison
text: Regression loss form and hyperparameter. Table 1 compares three forms of KLD-based regression loss on HRSC2016, including Dkl, f(Dkl) and Lreg(f(Dkl), τ). Due to extreme sensitivity to large
errors, the performance of Dkl is extremely poor, only 0.20%. Through a simple nonlinear linear transformation, the performance can be increased to 82.96% and 83.23% corresponding to sqrt and log. We further perform a detailed hyperparameter experiment on the loss Lreg proposed in this paper, and the performance reaches the optimal when τ = 1, f(Dkl) = log(Dkl + 1), about 85.25%. Keeping the same loss pattern, we compare six KLD-based distance functions in Table 2, and conclude that the asymmetry of KLD does not have much impact on performance. In subsequent experiments, we use Lreg(log(Dkl(Np||Nt)), 1) as the basic setting. Ablation study of normalization. As mentioned above, the use of Eq. 18 is to smooth its excessively rapid growth trend and play a role of normalization. This extra normalization questions if the KLD is actually contributing or simply produces noise in the results. In order to further prove that our method is indeed effective, we also perform a normalization operation on the Smooth L1 loss to eliminate the interference caused by normalization. As shown in Table 3, there is a significant drop in performance after using the normalization. The above experimental results prove that the effectiveness of KLD does not come from Eq. 18.
High-precision detection experiment. We expect that the designed rotation regression loss can show advantages in high-precision detection. Table 4 shows the comparison of the high-precision detection results of three different regression losses using Smooth L1, GWD and KLD on different datasets and different detectors. For the HRSC2016 dataset containing a large number of ship with large aspect ratios, GWD-based RetinaNet has a 11.89% improvement over Smooth L1 on AP75, KLD even gets a 23.97% gain. Even with a stronger R3Det detector, KLD and GWD still increased by 33.96% and 22.46% in AP75, and 15.22% and 9.89% in AP50:95. The same experimental conclusion are also reflected in the other two scene text datasets MASR-TF500 and ICDAR2015, which is KLD > GWD > Smooth L1. In general, the self-modulation optimization mechanism has a significant help for high-precision detection. For a more intuitive comparison, we visually compare these three regression losses, as shown in Figure 2. Since the center point (x, y) parameters in Smooth L1 Loss and GWD are independently optimized, their prediction results are slightly shifted. In contrast, the KLD-based prediction results are closer to the object boundary and show strong robustness in dense scenes. Similarly, GWD-based or KLD-based model has more accurate angle prediction capabilities than Smooth L1-based model due to their angle parameters (θ) are not independently optimized.
Ablation study on more datasets. To make the results more credible, we continue to verify on the other five datasets, as shown in Table 5. The improvement of KLD on the three data sets of MLT, UCAS-AOD and DOTA-v1.0 is still considerable, with an increase of 9.17%, 1.58%, and 5.55% respectively. Note that for DOTA-v1.5 and DOTA-v2.0, which contain a large number of small objects (less than 10 pixels), KLD has achieved significant gains of 3.63% and 3.53%.
Comparison of peer methods. Table 6 compares the six peer techniques, including IoU-Smooth L1 Loss [3], Modulated loss [45], RIL [34], CSL [4, 47], DCL [46], and GWD [5] on DOTA-v1.0. For fairness, these methods are all implemented on the same baseline method, and are trained and tested under the same environment and hyperparameters. We detail the accuracy of the seven categories,
including large aspect ratio (e.g. BR, SV, LV, SH, HA) and square-like object (e.g. ST, RD), which can better reflect the real-world challenges and advantages of our method. Without bells and whistles, the combination of RetinaNet and KLD directly surpasses R3Det (71.28% vs. 70.66% in AP50 and 69.41% vs. 68.31% in 7-AP50). Even combined with R3Det, KLD can still further improve performance of the large aspect ratio object (2.82% in 7-AP50) and high-precision detection (6.07% in AP75 and 3.65% AP50:95). KLD-based method shows the best performer in almost all indicators. Similar conclusions can still be drawn on the more challenging datasets (DOTA-v1.5 and DOTA-v2.0), which contain more data and tiny object (less than 10 pixels).
Horizontal detection verification. As analyzed by Eq. 16, KLD can be degenerated into the common regression loss in horizontal detection task. Table 7 compares the regression loss Smooth L1 and IoU/GIoU for horizontal detection with the proposed regression loss KLD on MS COCO [48] dataset. The results show that our KLD is not worse than other losses on the Faster RCNN [8], RetinaNet [10] and FCOS [21], and even has an improvement of 0.6% on RetinaNet. The ground truth for rotation detection is the minimum circumscribed rectangle, which means that ground truth can well reflect the true scale and direction information of the object. The “horizontal special case” described in this paper also meets the above requirements, the horizontal circumscribed rectangle is equal to the minimum circumscribed rectangle at this time. Although the ground truth of the COCO is a horizontal box, it is not the minimum circumscribed rectangle, which means that it loses the direction information and accurate scale information of the object. For example, a baseball bat placed obliquely in the image, the height and width of its horizontal circumscribed rectangle do not represent the height and width of the object itself. This causes that when KLD is applied to the COCO, the optimization mechanism of KLD that dynamically adjusts the angle gradient according to the aspect ratio is meaningless, which affects the improvement of the final performance. In general, this is a defect in the dataset annotation itself, not that KLD is not good enough. In fact, it is inappropriate to use the COCO to discuss θ = 0◦, because the COCO discards θ parameter. In addition, θ = 0◦ describes the instances in the horizontal position, but not mean all instances of the dataset are in a horizontal position. This paper uses COCO to discuss the “horizontal special case” to express that even if the dataset has certain labeling defects, KLD can have certain effects. After all, it is difficult to observe the performance improvement of all horizontal objects on the rotating dataset.

heading: 4.3 Comparisons with the State-of-the-Art Methods
text: The evaluation is performed on the DOTA, which contains a considerable number of categories, complexity scenes. Our single-scale model RetinaNet-KLD-R50 and R3Det-KLD-R50 achieve 75.28% and 77.36% respectively. They outperform multi-scale models as shown in Table 8. With large backbone and multi-scale testing, our method further achieves state-of-the-art accuracy 80.63%.

heading: 5 Discussions
text: Limitations. Despite the theoretical grounds and the promising experimental justifications, our method has an obvious limitation that it cannot be directly applied to quadrilateral detection [34, 45].
Potential negative societal impacts. Our findings provides a simple regression loss for highprecision rotation detection. However, our research may be applied to some sensitive fields, such as remote sensing, aviation, and unmanned aerial vehicles.
Conclusion. Departure from the vast existing literature in object detection, in this paper we have designed a new regression loss for rotation detection from scratch and consider the popular horizontal detection as its special case. Specifically, we calculate the KLD between the Gaussian distributions corresponding to the rotated bounding box as the regression loss, and we find that in the learning procedure guided by the KLD loss, the gradient of the parameters can be dynamically adjusted according to the characteristics of the object which is a desirable property for robust object detection, regardless its rotation, size and aspect ratio etc. We also proved that KLD has scale invariance, which is crucial for detection tasks. Interestingly, we have shown that KLD can be degenerated into the currently commonly used ln-norm loss in the horizontal detection task. Extensive experimental results across different detectors and datasets show the effectiveness of our approach.
Acknowledgments and Disclosure of Funding
This work was partly supported by National Key Research and Development Program of China (2018AAA0100704), Shanghai Municipal Science and Technology Major Project (2021SHZDZX0102) and NSFC (U20B2068, 72061127003, 61972250). Xue Yang was also partly supported by Wu Wen Jun Honorary Doctoral Scholarship, AI Institute, Shanghai Jiao Tong University, Shanghai, China.

Review summaries:
The paper proposes an algorithm that can detect slender objects, like text in images or boats in aerial images. The work mainly focuses on the bounding box regression task and includes a parameter for box rotation. Bounding boxes are represented as Gaussian functions instead of axis-aligned boxes. The scale invariant KL-divergence loss is applied to the regression task. An analysis of the loss function proclaims that the computation of gradients, during optimization, are adjusted with respect to the bounding box parameters.

This paper presents a new design of the regression loss function for oriented object detection. When modeling the rotated bounding boxes as 2D Gaussian distributions, the authors introduced the Kullback-Leibler Divergence for oriented bounding boxes regression as an alternative to the commonly-used Smooth-l1 loss function. The experimental results show that the proposed loss function could obtain better performance on several benchmarks. 

This paper focuses on learning high precision bounding boxes for rotated object detections. Instead of directly predicting the size of the box, they turn the box into a gaussian distribution and use the KL divergence between the gaussian distributions as the regression loss. They show that KLD is modulated where the optimization of one parameter will be affected by other hyper-parameters. For example, the gradients for the orientation involve the width and height of a box. Experiments show that the proposed loss function significantly improve the performance of a detector on different aerial image datasets and scene text datasets.