Learning High-Precision Bounding Box for Rotated Object Detection via Kullback-Leibler Divergence

1 Introduction . As a fundamental building block for visual analysis across aerial images , scene text etc. , rotated object detection has recently been developed rapidly [ 1 , 2 , 3 , 4 , 5 , 6 ] , which benefit themselves from the well-established horizontal detection approaches [ 7 , 8 , 9 , 10 , 11 ] . Specifically , many works [ 12 , 13 , 14 , 15 ] build themselves upon the previously established horizontal detection pipeline from an inductive perspective , as shown in Figure 1 ( a ) . However , these detectors are often unable to cope with challenging scenes well due to the limitations of current regression loss , such as large aspect ratio objects , dense scenes , etc. , resulting in obvious disadvantages in high-precision detection . ∗Part of the work was done during an internship at Huawei Inc. †Correspondence author is Junchi Yan . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . ( a ) Previous methods follow the induction paradigm from special horizontal to general rotated detection . ( b ) Our proposed method adopts a deduction methodology from general rotated to special horizontal detection . Figure 1 : Methodological road-map difference between horizontal detection ( special case ) and rotation detection ( general case ) in the previous methods [ 1 , 12 , 13 , 14 , 15 ] and the proposed method . In this paper , we take a step back , and aim to develop ( from a deductive perspective ) a unified regression framework for rotation detection and its special case : horizontal detection . In fact , our new framework enjoys a coherent property that it can be degenerated into the current commonly used regression loss ( e.g . ln-norm ) in special cases ( horizontal detection ) , as shown in Figure 1 ( b ) . For a devising a rotation regression loss for high-precision rotation detection , one important observation is that the importance of different parameters to different types of objects can vary . For example , the angle parameter ( θ ) and the center point parameter ( x , y ) are important for large aspect ratio objects and small objects , respectively . In another word , it is conjectured that regression loss should be self-modulated during the learning process and calls for more dynamic optimization strategy . Inspired by the above ideas , we first convert the rotated bounding box B ( x , y , h , w , θ ) into a 2-D Gaussian distribution N ( µ , Σ ) . As a standard distance metric , we then use the Kullback-Leibler Divergence ( KLD ) [ 16 ] to calculate the distribution distance between the predicted bounding box and ground truth as the regression loss . We compare KLD with Smooth L1 loss [ 7 ] and another distance metric , Gaussian Wasserstein Distance ( GWD ) [ 5 , 17 ] , and find that KLD has a more complete parameter optimization mechanism . In particular , by analyzing the gradient of the parameters during learning , we show that the optimization of one parameter will be affected by other parameters ( as the gradient weight ) . It means that the model will adaptively adjust the optimization strategy given a specific configuration of an object for detection , as shown can lead to excellent performance in high-precision detection . In addition , KLD is proven scale invariant , which is an important property that Smooth L1 loss and GWD do not possess . As the horizontal bounding box is a special case of the rotated bounding box , we show that KLD can also be degenerated into the ln-norm loss as commonly used in existing horizontal detection pipeline . The highlights of this paper are four-folds : 1 ) Differing from the dominant existing practices that build rotation detectors heavily upon the horizontal detectors , we develop new rotation detection loss from scratch and show that it is coherent with existing horizontal detection protocol in its degenerated case for horizontal detection . 2 ) To achieve a more principled measurement between the prediction and ground truth , instead of computing the difference for each physically-meaningful parameter related to the bounding box which are in different scales and units , we innovatively convert the regression loss of rotation detection into the KLD of two 2-D Gaussian distributions , leading to a clean and coherent regression loss . 3 ) Through the gradient analysis of each parameter in KLD , we further find that the self-modulated optimization mechanism of KLD greatly promotes the improvement of high-precision detection , which verify the advantage of our loss design . More importantly , we have theoretically shown ( in appendix ) that KLD is scale invariant for detection , which is crucial for the rotation cases . 4 ) Extensive experimental results on seven public datasets and two popular detectors show the effectiveness of our approach , which achieves new state-of-the-art performance for rotation detection . The source codes [ 18 ] are made public available . 2 Background . We first generally discuss the related works on both horizontal and rotated object detection . Then we summarize the current design paradigm of rotation regression loss from two kinds of methodologies , as shown in Figure 1 : one is inductive that tries to develop the general rotation detection from the special and classic horizontal detection pipeline . While the other is deductive that aims to devise a general rotation detection pipeline with horizontal detection as its special case . 2.1 Related Works . Horizontal object detection . Horizontal object detection which covers most existing detection literature , normally uses a horizontal bounding box to represent the object . The mainstream classical object detection algorithms can be roughly divided according to the following standards : Two [ 7 , 8 , 9 , 11 ] or Single-stage [ 10 , 19 , 20 ] object detection , Anchor-free [ 21 , 22 , 23 ] or Anchor-based [ 8 , 9 , 10 ] object detection and CNN [ 8 , 10 , 21 ] or Transformer-based [ 24 , 25 ] object detection . Although the pipelines may vary , the mainstream regression loss often uses the popular ln-norm loss ( such as smooth L1 loss ) or IoU-based loss ( such as GIoU [ 26 ] , and DIoU [ 27 ] ) . These abovementioned detectors have also been widely used in other scenarios and have achieved satisfactory performance . However , horizontal detectors do not provide accurate orientation and scale information . Rotated object detection . Recent advances in rotation detection [ 3 , 4 , 12 , 14 , 28 ] are mainly driven by adapting the horizontal object detectors with rotated bounding boxes to represent multi-oriented objects . To accurately predict the rotated bounding box , most rotation detection methods extend the ln-norm [ 12 , 15 , 29 , 30 , 31 ] used in horizontal detection , or construct a differentiable approximate IoU loss [ 3 , 5 , 32 ] . From scratch , we try to change the design of rotation regression loss from induction paradigm to deduction methodology , which in fact is a generalization to the horizontal case . In the following , we describe the existing works from the induction and deduction methodologies . 2.2 Inductive Thinking of Loss Design : from Special Horizon to General Rotation Detection . Regression loss is a vital part of most current object detection algorithms . For horizontal bounding box regression , the model [ 7 , 8 , 9 , 10 , 11 ] mainly outputs four items for location and size : tpx = xp − xa wa , tpy = yp − ya ha , tpw = ln ( wp wa ) , tph = ln ( hp ha ) ( 1 ) to match the four targets from the ground truth ttx = xt − xa wa , tty = yt − ya ha , ttw = ln ( wt wa ) , tth = ln ( ht ha ) ( 2 ) where x , y , h , w denote the center coordinates , height and width , respectively . Variables xt , xa , xp are for the ground-truth box , anchor box , and predicted box , respectively ( likewise for y , w , h ) . Extending the above horizontal case , existing rotation detection models [ 1 , 12 , 13 , 14 , 15 ] also use regression loss which simply involves an extra angle parameter θ : tpθ = f ( θp − θa ) , t t θ = f ( θt − θa ) ( 3 ) where f ( · ) is used to deal with angular periodicity , such as trigonometric functions , modulo , etc . The overall regression loss for rotation detection is : Lreg = ln-norm ( ∆tx , ∆ty , ∆tw , ∆th , ∆tθ ) ( 4 ) where ∆tx = tpx− ttx = ∆xwa , ∆ty = t p y− tty = ∆y ha , ∆tw = tpw− ttw = ln ( wp/wt ) , ∆th = t p h− tth = ln ( hp/ht ) , and ∆tθ = t p θ − ttθ = ∆θ . It can be seen that parameters are optimized independently , making the loss ( or detection accuracy ) sensitive to the under-fitting of any of the parameters . This mechanism is fatal to high-precision detection . Taking the left side of Figure 2 as an example , the detection result based on the Smooth L1 loss often shows the deviation of the center point or angle . Moreover , different types of objects have different sensitivity to these five parameters . For example , the angle parameter is very important for detecting objects with large aspect ratios . This requires to select an appropriate set of weights given a specific single object sample during the training , which is nontrivial or even unrealistic . 2.3 Deductive Thinking of Loss Design : from General Rotation to Special Horizon Detection . To break the original inductive design paradigm , we adopt deductive paradigm to construct more accurate rotation regression loss . Here we rephrase the main idea in the recent work [ 5 ] , which converts a arbitrary-oriented bounding box B ( x , y , h , w , θ ) into a 2-D Gaussian N ( µ , Σ ) , as illustracted in Figure 3 . Then the distance between two Gaussian is calculated as the final loss . Specifically , the conversion is : µ = ( x , y ) > Σ1/2 =RΛR > = ( cos θ − sin θ sin θ cos θ ) ( w 2 0 0 h 2 ) ( cos θ sin θ − sin θ cos θ ) = ( w 2 cos2 θ + h 2 sin2 θ w−h 2 cos θ sin θ w−h 2 cos θ sin θ w 2 sin2 θ + h 2 cos2 θ ) ( 5 ) where R represents the rotation matrix , and Λ represents the diagonal matrix of eigenvalues . The recent work [ 5 ] analyzes that the introduction of N ( µ , Σ ) can solve the inconsistency between metric and loss , boundary discontinuity and square-like problem . On this basis , we further study how to design high-precision detection regression loss through new parameter space . Our view is that the self-modulated mechanism is positively correlated with the final high-precision performance . Gaussian Wasserstein Distance . The Wasserstein distance [ 5 , 17 ] between two probability measures Xp ∼ Np ( µp , Σp ) and Xt ∼ Nt ( µt , Σt ) expressed as : Dw ( Np , Nt ) 2 = ‖µp − µt‖22︸ ︷︷ ︸ center distance + Tr ( Σp + Σt − 2 ( Σ1/2p ΣtΣ1/2p ) 1/2 ) ︸ ︷︷ ︸ coupling terms about hp , wp and θp ( 6 ) Eq . 6 shows that the Gaussian Wasserstein Distance ( GWD ) is mainly divided into two parts : the distance between the center points ( x , y ) and the coupling terms about h , w and θ . Accordingly , the regression loss based on GWD can be regarded as a semi-coupled loss . Although GWD can greatly improve the performance of high-precision rotation detection due to the coupling between part of the parameters , the independent optimization of the center point make the detection result slightly shifted ( see Figure 2 ) . Note that GWD is not scale invariant , which is not detection friendly . When all the boxes are horizontal ( θ = 0◦ ) , Eq . 6 can be further simplified : Dhw ( Np , Nt ) 2 =‖µp − µt‖22 + ‖Σ1/2p −Σ 1/2 t ‖ 2 F = ( xp − xt ) 2 + ( yp − yt ) 2 + ( ( wp − wt ) 2 + ( hp − ht ) 2 ) /4 =l2-norm ( ∆x , ∆y , ∆w/2 , ∆h/2 ) ( 7 ) where ‖ · ‖F is the Frobenius norm . Although Eq . 7 can still be used as the regression loss of horizontal detection , Eq . 4 and 7 are not completely consistent . Although GWD scheme has played a preliminary exploration of the deductive paradigm , it does not focus on achieving high-precision detection and scale invariance . In the following , we will propose our new approach based on the Kullback-Leibler divergence ( KLD ) [ 16 ] .

Summary: This paper focuses on learning high precision bounding boxes for rotated object detections. Instead of directly predicting the size of the box, they turn the box into a gaussian distribution and use the KL divergence between the gaussian distributions as the regression loss. They show that KLD is modulated where the optimization of one parameter will be affected by other hyper-parameters. For example, the gradients for the orientation involve the width and height of a box. Experiments show that the proposed loss function significantly improve the performance of a detector on different aerial image datasets and scene text datasets.